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ABSTRACT 

This final report discusses the activities and outcomes of a 
project that addressed three challenges to improving outcomes for young 
children with disabilities : (1) a current and thorough synthesis of the 

knowledge base, (2) the effective translation and dissemination of this 
information for families, practitioners, and administrators, and (3) an 
understanding of policy and systems change support necessary to implement and 
sustain high quality services. In Phase 1, the knowledge base was synthesized 
by critically reviewing the extant literature, gathering the subjective views 
of 388 stakeholders, and synthesizing this knowledge base into 240 
recommended practices. In Phase 2, the project developed and disseminated 
products in preferred formats for maximum suitability, prepared and field- 
tested products, disseminated the recommended practices nationwide, and 
provided regional and national training. In Phase 3, the project identified 
and incorporated training and systems change strategies to establish 
sustainable change in the quality of services for young children with 
disabilities and their families. The project also prepared six peer reviewed 
manuscripts for publication and video and Web-based products to accompany the 
print products produced in Phase 2. Appendices include: Figures; Literature 
Review Coding Forms; Manuscripts; Products, Web Site and Training; and Focus 
Group Recommendations . (Contains 17 references.) (CR) 
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SYNTHESIZING AND DISSEMINATING RECOMMENDED PRACTICES IN EARLY 



INTERVENTION/EARLY CHILDHOOD SPECIAL EDUCATION 

I. EXECUTIVE SUMMARY 

Research in EI/ECSE has reported practices that can improve outcomes for young 
children with disabilities (DEC, 1993; Guralnick, 1997). However, the widespread adoption of 
these recommended practices has been hampered by three challenges: a) a current and thorough 
synthesis of the knowledge base, b) the effective translation and dissemination of this 
information for families, practitioners and administrators and, c) an understanding of policy and 
systems change supports necessary to implement and sustain high quality services. 

This project addressed all three of these challenges to improving outcomes for children 
with disabilities, ages birth through five years, through three objectives or phases. Phase 1: 
Synthesized the knowledge base by critically reviewing the extant literature, gathering the 
subjective views of stakeholders and synthesizing this knowledge base into 240 recommended 
practices. Phase 2: Developed and disseminated products in preferred formats for maximum 
usability; prepared and field-tested products; disseminated the recommended practices 
nationwide; and provided regional and national training. Phase 3: Identified and incorporated 
training and systems change strategies to establish sustainable change in the quality of services 
for young children with disabilities and their families requiring on-going and purposeful 
strategies. By building partnerships with two local programs and obtaining a model 
demonstration grant from OSEP, we have continued our efforts to create systems change. We 
have prepared six peer reviewed manuscripts for publication and video and web based products 
to accompany the print products produced in Phase 2; disseminated systems change and 
administrative strategies necessary for the sustainable adoption of recommended practices; and 



collaborated with national stakeholder groups to provide training and information to their 
constituents who represent key consumer groups. 
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III. PROJECT OVERVIEW & CHALLENGES 



This project has been a collaborative effort between the Council for Exceptional 
Children’s Division for Early Childhood (DEC), the University of Colorado-Denver, the 
Louisiana State University Medical School, and the University of Wisconsin-Milwaukee to 
update, expand and validate the DEC Recommended Practices in Early Intervention and Early 
Childhood Special Education. The second goal has been to disseminate the practices and to 
conduct systems change and training activities to improve the likelihood that the practices are 
adopted by local programs to improve services to young children with disabilities and their 
families. 

This report details the project objectives, activities, accomplishments and outcomes. In 
addition, administrative accomplishments included: fiscal and personnel arrangements, executed 
subcontracts to the Louisiana State University Medical Center (LSUMC) for the literature review 
activities, and with the University of Wisconsin-Milwaukee (UWM) for the field validation 
study. We established a cross-site Management Team consisting of the P.I. (Smith), the three 
investigators (Sexton, McLean, Sandall), the strand chair coordinator (Odom), and the three 
methodology consultants (Snyder, Strain, Thompson). Management by objectives procedures 
were put into place, regular Management Team conference calls held, at least two face-to-face 
Management Team meetings held yearly, attended the OSEP Research Directors meetings, and 
all fiscal/budget matters were monitored. All the experts in the field that had offered in the 
proposal to chair recommended practices strands, agreed to carry out that activity - Bruder, 
Wolery, McWilliam, Santos & Thomas, Salisbury, Stremmel, Harbin, Miller & Stayton, Dunst & 
Trivette, Bagnato & Neisworth (see Appendix A). All strand chairs and consultants contracts 
were executed. 
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Two unforeseen factors occurred affecting our timelines and some proposed procedures. 
First, the number of research articles that met our criteria for inclusion far exceeded our 
expectations. Therefore, the number of articles to be reviewed and coded required more volunteer 
coders than we had anticipated, and a longer time frame for the literature review activities. We 
had proposed to be completed with the literature review phase by the beginning of the third 
quarter of Year 01. In order to achieve this, we requested supplemental funding to hire more staff 
and to provide stipends to coders in order to enforce a short turn around. Our request was denied 
and we were advised to modify the work scope in some way as an option. Therefore, rather than 
delete key planned activities which we felt would jeopardize the integrity of the literature review 
and synthesis, we, modified four activities and the time line to allow for completion of the 
literature review by the fourth quarter. The four modifications were: 1) to include articles that 
were published through 1998 vs. 1999; 2) not to compute effect sizes as part of the coding 
process, 3) to conduct the synthesis by the Investigators reviewing all recommended practices in 
a strand and combining where possible to produce a feasible number of practices for the field to 
implement, then having the strand chairs validate the synthesis (vs. a delphi technique); 4) initial 
data entry was confined to the limits of the originally proposed data entry vs. an expanded data 
set capable from the coding sheets. Data from the generic coding sheets (not related to the 
specifics of the study methodology) and the recommended practices were entered for all articles 
returned by coders. These recommendations were made by the project’s Management Team as 
not necessary to the integrity of the project and yet impacting on the time frame. However, the 
project subsequently received supplemental funding to conduct analyses of the expanded data set 
which was achieved and is reported in this report. 

The second unforeseen occurrence was the sudden death of Dr. David Sexton in 
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November of 1999. Dr. Sexton was the Principal Investigator on the LSUMC subcontract at the 
time of his death. The focus of the LSU subcontract was the literature review. Completion of the 
data entry was affected by approximately one month. Additional data entry on the second level of 
review (specific of the study methodology) was postponed. In February, 2000, Dr. Pat Snyder 
was named Principal Investigator for the LSUMC subcontract and work resumed on the second 
level data entry. While these events affected the literature review by several months, all other 
major activities have occurred on time. 

IV. CONCEPTUAL FRAMEWORK 

For over 30 years, literature in Early Intervention/Early Childhood Special Education 
(EI/ECSE) has reported strategies, methods and tools for improving the developmental and 
educational outcomes for young children with disabilities (Guralnick, 1997; Meisels & Shonkoff, 
1990; Tjossem, 1976; White, Mastropieri, & Casto, 1984). Previous efforts to synthesize the 
EI/ECSE knowledge base have been limited. For example, in the 1980s researchers employed 
meta-analytic techniques to synthesize the early intervention efficacy literature (Casto & 
Mastroprieri, 1986; Casto, White, & Taylor, 1983); however, large bodies of relevant research 
(e.g., single-subject and qualitative studies) were not included in the synthesis (Dunst & Snyder, 
1986; Strain & Smith, 1986). 

Another effort to summarize the EI/ECSE knowledge base, sponsored by the Division for 
Early Childhood (DEC) of the Council for Exceptional Children (CEC), was begun in 1991. 
Recommended practices in fourteen areas, called “strands”, were generated by groups composed 
of researchers, practitioners, representatives from higher education, and families. These practices 
were subsequently validated by a national sample of 500 individuals representing these 
stakeholder groups. The resulting list of 405 practices, first published in 1 993 in a DEC 
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document (DEC Task Force on Recommended Practices, 1993) was later followed by a book 
(Odom & McLean, 1996) that further explained the practices. Although this effort was generally 
well received, a number of concerns with the 1993 effort signaled the need for a more systematic 
and comprehensive synthesis of research and practices in EI/ECSE (Garland, 1995; Wolery, 
1995). First, the 1993 DEC practices were derived solely from stakeholder focus groups; no 
systematic review of the scientific literature was conducted. Therefore, many of the original 405 
practices recommended by DEC were not based on empirical research but on the values and 
experiences of individuals involved in the work groups. Subsequently, readers of the 1993 
document or the 1996 book are unable to determine which practices have empirical support and 
which are values-based. Second, the reports and book generated from this effort were not 
produced in user-friendly formats and consumers reported there were too many practices to be 
useful in quality improvement efforts. Third, the recommendations only addressed direct services 
and personnel development. There were no recommendations for administration or policy 
support necessary for the delivery of quality direct services. Finally, the landscape of EI/ECSE 
has changed significantly since DEC’s work on the original recommended practices project. The 
number of preschool programs has grown dramatically, empirical research has expanded, 
services are provided in more inclusive settings, partnerships with general early childhood 
education have grown, and all states are participating in Part C of the Individuals with 
Disabilities Education Act (IDEA) (birth to three programs). 

In addition to updating the practices, it was recognized that there was a need to use more 
rigorous methods for producing a set of recommended practices in order to increase the 
likelihood of their adoption and implementation at the child, family, and systems levels. These 
improved methods included: reviewing the research literature for practices that result in 
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improved outcomes; conducting focus groups of parents, practitioners, administrators, and 
scientists to ascertain their beliefs and values about practices that result in improved outcomes; 
synthesizing these sources of information; producing recommendations for “indirect” practices or 
supports such as personnel preparation and policy and systems change activities necessary for the 
implementation of recommended practices with children and families; and increasing the 
awareness and use of the recommended practices by disseminating the practices through user- 
friendly products, engaging in training activities, and collaborating with partner national 
organizations for dissemination to key stakeholder groups such as families, early childhood 
educators, and administrators. 

V. METHODS AND PARTICIPANTS 

The procedures used to compile the recommended practices were accomplished in three 
stages. Stage One involved identifying recommended practices through focus groups and a 
comprehensive literature review. In Stage Two, practices developed in the focus groups were 
synthesized with practices generated from the literature review. Stage Three involved a field 
validation of the synthesized practices. 

The project employed a mixed-method design with an emphasis on integrative 
approaches. From the beginning, a Management Team of eight researchers was established to 
ensure that all decisions were subjected to expert review and represented a consensus of opinion. 

Stage One: Identifying Recommended Practices 

Experience-based practices: focus groups . In 1998-1999, focus groups were conducted in an 
effort to gather practices based upon participant experiences. Four categories of focus groups 
were held to generate the recommended practices that stakeholders believed were important for 
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improved outcomes for children and families. Focus groups were organized by role (a) experts by 
topic strand, (b) families, (c) administrators, and (d) practitioners. 

The scientific expert focus groups were organized by nine topic strands and were 
conducted during the 1 998 DEC annual conference in Chicago by strand chairs selected for their 
extensive and recognized work in their particular area. The nine strand categories were 
determined a priori according to whether they reflected direct services or indirect supports. The 
six direct services strands were child- focused practices, cultural/linguistic sensitivity, family- 
based practices, interdisciplinary models, learning environments, and technology applications. 
The three indirect support strands were personnel development, policy/procedures, and systems 
change/leadership. The strand chairs selected up to 10 individuals with expertise in their area to 
serve as focus group members (Greenbaum, 1987). 

The Management Team met with the strand chairs before the focus groups to review the 
purpose of the groups and to establish general procedures for generating practices. Strand chairs 
developed a definition of their strand prior to conducting the focus groups to help guide the 
group and provide a focus for the discussion. Agreement on the format for stating practices was 
reached. Finally, follow-up procedures and timelines for conducting the focus groups were 
determined. Each focus group lasted approximately 2 hours and was audio taped. The direct- 
services focus group members were asked to list the practices in the strand topic area that have 
been shown to produce positive outcomes for children birth through 5 years or their families. The 
focus groups for the indirect supports of systems change, policies and procedures, and personnel 
development were asked what practices in these areas support the delivery of direct services. The 
practices generated by the focus groups were subsequently compiled by the strand chairs using 
the audio tapes to supplement table notes or to produce transcripts. The practices were then 
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mailed to each member of the focus group for review and verification and then submitted to the 
Management Team. After finalizing the lists of practices, the Management Team decided to add 
a strand on assessment, combine the policy/procedure strand with the systems change strand, and 
synthesize the learning environments and cultural/linguistic practices across all other strands and 
practices. This Management Team decision, which was also supported by the strand chairs, was 
based on a consensus opinion that assessment issues had not been addressed adequately in some 
strands but were redundant across others. Secondly, it was decided that practices in the learning 
environment and cultural linguistic strands were identified and appropriate across all other 
strands. This initial experience based synthesis resulted in seven final strands of recommended 
practices: five direct services— assessment, child-focused practices, family-based practices, 
interdisciplinary models, technology applications, and two indirect supports— personnel 
preparation, and policies, procedures and systems change. The resulting practices generated by 
the scientific expert focus groups formed the corpus of information that was ultimately judged 
against the support generated by the literature and the stakeholder focus groups. 

The categories of stakeholders for the three experience based focus groups were families, 
practitioners, and administrators. These stakeholder focus groups were also conducted during the 
1998 DEC conference and followed the same procedures described above. Following each focus 
group, the recommended practices generated by the group were summarized by the focus group 
facilitator who was one of the project investigators. This summary was sent to each focus group 
member for review and verification. The practices from each focus group were subsequently 
organized by topic strand with all of the recommendations from the stakeholder and scientific 
expert focus groups categorized, depending on their topic, and placed into one of the seven 
strands (e.g., child-focused, family-based, etc.). 
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Research-based practices: literature review . Empirically based research published in peer- 



reviewed professional journals from 1990 through 1998 was included in the literature review. 
Forty-eight journals relevant to the field of EI/ECSE were selected and reviewed. Of these, forty 
five were determined to have articles that contained research findings with implications for 
practice (see table 1). First, the tables of contents for all 48 journals from 1990 to 1998 were 
reviewed by project staff. Second, the article abstract for each article was reviewed to determine 
if the article met the criteria for inclusion in the study. The criteria were that the article must be 
an empirical study of an intervention provided to children with disabilities, birth through 5 or 
their families or those who serve them. Each article that passed this initial screening was then 
analyzed for major aspects of research design and results. Analysis of the articles was 
accomplished by having individual coders read articles and complete two coding sheets for each 
article. 

A generic coding sheet (Form A) was used uniformly across all articles. This coding sheet 
and was designed to record the following information: article title, authors, journal, year, general 
description of participants (children with disabilities, families of children with disabilities, etc.), 
gender of participants, mean age, ethnicity, disability, severity level of disability, educational 
level of family members, income level of family members, and study setting. A specialized 
coding sheet (Form B) designed specifically to highlight additional information about research 
rigor based on the method used in the articles was also completed. Coding sheets for group 
quantitative design, single subject design, descriptive/survey design, qualitative design, and 
mixed method design were used. The specialized coding sheets were developed by the project’s 
methodology consultants to ensure that each article was appropriately reviewed according to the 
research method used in the article. The information on the specialized coding sheets included 
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variable identification, research design features, sample, setting, outcome measures, duration of 
intervention, findings, recommended practice(s) supported by the study, and the strand that the 
recommended practice reflected. A main purpose of the specialized coding sheets was to help the 
coder evaluate the scientific rigor of the article and subsequently lead the reviewer to the 
recommended practice(s) supported by the article (see Appendix B for coding sheets). 

Forty-two first stage coders were trained, and inter-coder agreement was established with 
a methodology consultant. Coders had to read and code an article(s) and establish an inter-coder 
agreement level of at least 85% with a methodology consultant before formal article coding 
began. Following training, coders were assigned articles to read and code according to the 
generic and specialized coding sheets. Twenty-nine additional, or second stage, coders read 
approximately one-third of the articles (n = 360) that had been coded by a first stage coder and 
independently coded the articles as to the recommended practice(s) and strand placement 
generated by the first coder. When discrepancies occurred between the two coders, the article 
was reviewed and read by two members of the Management Team and a consensus was reached 
about the recommended practice and strand placement. This occurred for less than 5% of the 
articles reviewed. 

The initial number of articles included in the review was 1,018. Fifty-four percent (n = 
550) of the articles reviewed reported a group quantitative design, 21% (n = 210) single subject, 
14% (n = 142) descriptive/survey, 11% (n = 109) qualitative, and 1% (n = 7) mixed method. 
After review, or as a result of the validation process, 183 articles were excluded because the 
research conducted did not lead to a recommended practice or after further review the article did 
not meet the original criteria for inclusion in the study, resulting in a final set of 835 articles. 
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Stage Two: Synthesis of Experience and Research 



Practices generated from the scientific expert focus groups served as a baseline of 
reference from which to examine the support available to individual practices from the empirical 
literature and from stakeholder focus groups. To accomplish this task, the stakeholder focus 
groups and the literature review were combined (synthesized) within each strand by the project 
investigators (See Appendix A). This synthesis combined similar practices, deleted duplications, 
and added new practices to subsume groups of similar practices. All decisions were made by 
consensus among the investigators. These lists of practices by strand were submitted to 
participants in several sessions at the 1999 DEC conference for review and comment and were 
subsequently reviewed by the strand chairs and project investigators for final verification. The 
number of practices after the synthesis was 250. 

Stage Three: Field Validation of Practices 

After synthesizing the recommended practices produced from the focus groups and 
literature review, the final list of 250 practices were subjected to a national field validation. The 
validation stage included two steps: (a) verification among experts, and (b) performance of a 
national survey described below. Verification among experts involved a review of the final list of 
practices and supporting articles by the strand chairs as well as a review by the project 
investigators. 

Second, the list of 250 practices was formatted into a questionnaire for field validation. 
Due to the large number of practices, two forms of the questionnaire were created by assigning 
odd-numbered items to Form A and even-numbered items to Form B. Each respondent received 
either Form A or Form B with 125 practices to validate. Respondents used a Likert-type scale 
with ratings of strongly agree, agree, disagree, strongly disagree , or undecided for whether the 



practice should be a recommended practice in Early Intervention/Early Childhood Special 
Education. Respondents also used a Likert-type scale to indicate how often the practice is used, 
rating frequently, sometimes, rarely, never, or undecided . 

DEC members, parents, and higher education professionals and administrators were the 
three groups that comprised the validation sample. Initially, the groups consisted of volunteers 
who responded to requests at the annual DEC conference and to a notice in Young Exceptional 
Children to participate in the field validation. Additional participants were identified to increase 
the sample. After including the volunteers and randomly selecting members from the DEC 
membership list, the first group consisted of 400 DEC members. The second group consisted of 
200 parents, which included the volunteers and individuals identified by the DEC Family 
Consortium and also individuals randomly selected from the mailing list of members of the 
Federation for Children with Special Needs. The third group was comprised of 200 individuals in 
higher education and administrative positions in Early Intervention/Early Childhood Special 
Education, which included volunteers and individuals randomly selected from the list of Part C 
and 619 Coordinators from each state. This resulted in a final list of 800 people participating in 
the validation study. 

Questionnaires were mailed to participants. The initial mailing to respondents was 
followed one month later by a postcard reminder with a second postcard reminder mailed 
approximately 1 month after the first. After 38 surveys were returned blank, the original sample 
of 800 people was later changed to 762. 

VI. RESEARCH FINDINGS 

The final number of recommended practices by strand is as follows: assessment 46, child- 
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focused 27, family-based 17, interdisciplinary models 19, technology applications 22, personnel 
preparation 66, and policies, procedures and systems change 43. Over 70% of the recommended 
practices on the final list are supported by some combination of the three sources, with over 65% 
of the practices having literature support from at least one empirically based research article. 
Experience-Based Practices: Focus Groups 

The stakeholder focus groups (family members, administrators, and practitioners) 
generated 145 recommended practices. As described previously, practices generated in the 
stakeholder focus groups were first grouped into the strand topics then synthesized with practices 
developed in the scientific expert focus groups. During the synthesis it was discovered that some 
of the practices generated in the stakeholder focus groups applied to more than one recommended 
practice on the final list. Subsequently, the total number of practices generated in the three 
stakeholder focus groups that were used to support the recommended practices on the final list 
was 158. After combining similar practices from the three focus groups and synthesizing these 
practices with the final list, 96 (40%) of the recommended practices on the final list were 
supported by practices that emerged from at least one of the stakeholder focus groups. Most of 
the practices generated by family members fall into the family-based and the policies, 
procedures, and systems change strands. Not surprisingly, the majority of practices generated by 
administrators fell into the policies, procedures, and systems change strand. Practitioners 
generated nearly twice the number of practices as did the family members and administrators, 
with the majority of practices falling in the policies, procedures, and systems change, family- 
based, and personnel preparation strands. 

Research-Based Practices: Literature Review 

The number of articles derived from the initial screening was 1,018. The final number of 
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articles included in the review was 835. Articles (183) were excluded by coders if the research 
conducted did not lead directly to a recommended practice, the article did not meet the original 
criteria for inclusion in the study, or a coder and a methodology consultant determined that the 
article did not meet the projects’ scientific standards to merit inclusion (e.g., case studies had 
been incorrectly determined to be single subject design studies). Of the 835 articles coded, 54% 
(n=450) employed a group quantitative design, 22% (n=184) used a single subject design, 15% 
(n=121) used descriptive/survey designs, 9% (n=lA) qualitative designs, and 1% (n= 6) used 
mixed methods. Table 2 shows the final number of articles with research findings interpreted by 
coders as having implications for recommended practices by research method. 

The total number of recommended practices derived from the 835 articles reviewed was 
1,453. This total number of recommended practices exceeds the number of articles reviewed 
because according to the coders, some studies generated evidence that produced more than one 
recommended practice. Table 3 shows the number and percent of articles according to the 
research methodology employed and the recommended practice strand assigned by the coder. For 
example, 399 articles supported the Child-Focused strand. One hundred and eighty-seven (47%) 
of those articles used a group quantitative method, 158 (40%) used a single subject design, 38 
(10%) a survey/ descriptive design, 12 (3%) qualitative approaches and 4 (1%) used mixed- 
method desiga Literature support for practices were noted 977 times across the seven strands 
and across the five different methodologies. This number is higher than the actual number of 
articles (835) because some articles contained more than one practice and at the same time 
supported two or more strands in the judgment of the coders. Further review of Table 3 shows 
that: (a) a disparity exists in the amount of empirical research that supports the various practice 
strands and, (b) regardless of practice strand, group quantitative methods were the predominant 
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research method employed during 1990-1998 in this EI/ECSE research. 

Table 4 displays the study methods employed by articles by year. For instance, group 
quantitative methods were used in 41 (54.7%) of the 1990 articles, while single subject and 
descriptive/survey designs were used in 14 (18.7%), qualitative in 6 (8.0%) and mixed methods 
were used in no articles that year. There is a general trend towards increased numbers of articles 
employing qualitative methodology over the 9 years. The use of single subject methodology 
increased from 1990 to 1998 but varied with each year. Group quantitative methodology remained 
the highest percentage method used across all years. 

Tables 5 through 8 depict analyses of the Form A data. Table 5 shows the number and 
percent of articles that reported the disability category of child study participants by year. The 
“other” category included coder comments such as: “at risk for developmental disability”, 
neurologically impaired”, “fetal alcohol syndrome”, “spina bifida”, “sick preemies”. The coders 
were instructed to code the disability category as defined by the article. Therefore, there were 
categories or subcategories that were not on the coding sheet. The coders coded these under 
“other” rather than make an interpretation. 

Table 6 shows the number and percent of articles reporting the level of severity of 
disability of child study participants by year. These data show that on average 53%-73% of 
articles report the severity of disability with the percent declining over time from 73% in 1990 to 
55% in 1998. 

Table 7 shows the number and percent of articles reporting the study setting. For the 
study setting variable, no explicit definitions were offered to coders. Instructions were to code 
what the author(s) reported as the setting of the study. The setting options for coders on Form A 
were: home, hospital/clinic, inclusive/integrated classroom, segregated classroom, or other. 
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Initially the “other” category contained nearly 40% of the responses. Two project staff reviewed 
the entries in the “other” category and agreed to (a) distinguish “survey/interview” (15.8%), and 
“not clear” entries (2.4%); (b) add “separate” to the segregated classroom category for entries 
that described settings that are segregated but not a classroom (“speech lab”, “OT room”, etc.); 

(c) create a new category of “experimental settings/separate from peers” for entries that described 
the study setting as “experimental setting”, “quiet room”, etc.; and, (d) add entries related to 
“community” to the “inclusive/integrated” category. Table 7 shows that nearly half of the 
research settings in this EI/ECSE literature between 1990-1998 were home or hospital/clinic. 
Less than 20% of studies were conducted in inclusive/integrated classrooms or community 
settings. 

Table 8 reports the number and percent of articles by year that reported income level and 
education level of families in the study. There do not appear to be any trends in these data. 
Income level is reported between 19-26% of the time and education level between 20-30% of the 
time in the EI/ECSE literature included in this effort. 

Analyses of data from the individual specialized forms (Forms B) are found in Appendix 
C which contains manuscripts submitted for publication and provide: 1) complete description of 
the literature review, 2) analysis of the single subject design research articles, 3) analysis of the 
group quantitative design research articles, 4) analysis of the qualitative design research articles, 
5) analysis of the interdisciplinary articles, and 6) analyses of the field validation study. 

Field Validation and Final Recommended Practices. 

A return rate of 51% (n = 388) was obtained (211 DEC members, 55 parents, and 122 
higher education/administrators). The criterion used to determine whether a practice should be 
considered to be a validated recommended practice was that more than 50% of the respondents to 



a particular item indicated strongly agree or agree in response to the statement “This is a 
recommended practice.” All 250 practices included in Form A and Form B of the survey met this 
criterion. Therefore, all the practices were validated as recommended practices. The validation 
survey also asked respondents to rate how often the practice is used. These data were analyzed 
using the criterion that 50% or more of the respondents rated a practice as occurring frequently 
which resulted in only 13 (5%) out of 250 practices meeting this criterion. The data were then 
analyzed using the criterion that 50% or more of the respondents rated a practice as occurring 
sometimes resulting in 227 (91%) practices meeting this criterion. When combined, 240 out of 
the 250 (96%) practices met this criterion. 

All 250 practices met the criteria for validation. A final editing by the investigators for 
consistency and redundancy of the validated practices resulted in 1 0 practices being considered 
redundant with others thus, resulting in a final list of 240 recommended practices in seven 
strands: Assessment, Child-Focused, Family-Based, Interdisciplinary Models, Technology 
Applications, Personnel Preparation, and Policies, Procedures and Systems (Sandall, McLean & 
Smith, 2000). 

VII. PROJECT IMPACT: Discussion, Products and Dissemination 

The data show which scientific method has been employed to study particular strands and 
specific practices. These data present the most comprehensive and systematic review of the 
EI/ECSE research literature to date about which topics and practices have been empirically 
studied and with which methodologies. 

Although this study is the most comprehensive of its kind to date, there are several 
limitations to the data. First, the criteria for selecting articles to be included in the review was 
limited to the 48 peer reviewed journals chosen by the management team and the strand chairs. 
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That is, books, chapters, and conference proceedings were not included in this review. Second, 
studies selected for coding were based on the judgments of two investigators who read abstracts 
in the 48 journals over the 9 years. Third, due to resource limitations, the coding of all articles 
was completed by volunteers. However, inter-coder reliability of at least 85% was established 
with all coders prior to article coding and inter-coder agreement checks for recommended 
practices and strand placement were completed for 1/3 of the coded articles. 

There are several important implications of the study. First, the EI/ECSE knowledge base 
is founded on a wealth of research. In less than 10 years, over 1000 peer reviewed articles have 
been published with over 3/4 of them containing implications for practice. However, most 
journals in the field are not reporting significant numbers of empirical studies as defined in this 
project related to children with disabilities birth through five. Only about 25% of the journals 
reported 20 or more empirical studies related to this population over the 9 years reviewed. 
Finally, it is evident that the field of EI/ECSE has benefited from the rich array of research 
methods available to study particular questions. 

The analyses of the literature base reveals several possible directions for the field. First, 
future studies would benefit from clarifying study participants, their disability, the severity of the 
disability, ethnicity and any other characteristics that may impact on the usefulness and 
replicability of research findings. Journals might suggest standard classifications such as the 
IDEA definitions of disability, categories of severity of disability, and categories of study 
settings. Second, where it may be germane to replicability of results, clarity of family and service 
provider characteristics should be described. Finally, there are areas of practice that have less 
empirical research support than others, including interdisciplinary models, technology 
applications and personnel preparation. 
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Following are some recommendations for future similar efforts to summarize the research 
base in EI/ECSE. Given the impetus to be evidence-based, these recommendations may lead to a 
more efficient method of review and an improved synthesis of the data-base. First, we 
recommend a review of the literature take place at least every 5 years. This might reduce the 
number of articles that need to be reviewed and coded. Second, such a review requires adequate 
resources. The efforts described in this paper are due to the many volunteer coders in the field. It 
is unlikely that this volunteer resource can be tapped indefinitely! Third, coding forms should 
include more standardization of terms and few open-ended entry options to improve clarity and 
consistency and reduce interpretation errors. 

This is the first large-scale effort to summarize the literature base in EI/ECSE across all 
research methods. Such efforts are important to continue so that researchers can improve 
methods and reporting of settings, variables, and findings; scientific journals can endeavor to 
encourage more precision in studies and manuscripts; and funders and researchers can be 
evidence-based in identifying gaps or areas needing further attention in the on-going quest to 
identify which services and supports are most effective for which children, families, or the 
personnel who serve them. 

Dissemination Activities: Products, Distribution of Products and Regional and National Training 

We have engaged in many effective dissemination activities including the production of 
user-friendly products, widely distributing the products, and providing national and regional 
training. First, we held focus groups of family members, administrators, practitioners and 
personnel trainers to learn in which formats they would prefer the recommended practices: text 
book, program level materials, videos, CD Rom, check lists, etc. All groups suggested check 
lists, videos program level instruments and the personnel trainers suggested guidelines for 
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training as well as modules. All groups discouraged high tech materials such as CD Rom, and 
other web based material as not cost effective at this time. 

Following these suggestions, we produced the following list of products (see Appendix 
D for copies of all products and flyer): 

• a text book or overview document that describes the project and the DEC Recommended 
Practices: DEC Recommended Practices in Early Intervention/Early Childhood Special 
Education (Sandall, McLean & Smith, 2000). 

• Second, we produced a program self-assessment guide: DEC Recommended Practices 
Program Assessment: improving practices for young children with special needs and their 
families (Hemmeter, Joseph, Smith & Sandall, 2001). 

• Third, we produced a video of the Child Focused practices: DEC Recommended Practices: 
Selected Strategies for Teaching Young Children with Special Needs (DEC, 2001). 

• Fourth, we are producing a guide for personnel trainers: Personnel Preparation in Early 
Childhood Special Education: Implementing the DEC Recommended Practices (Stayton, et. 
al., in press). 

• Fifth, a side-by-side comparison of the DEC Recommended Practices and the NAEYC 
Developmentally Appropriate Practices is in preparation 

• Sixth, check lists for administrators and parents are posted on the DEC web site: www.dec- 
sped.org 

• Six manuscripts have been submitted for publication in peer reviewed journals (see Appendix 
C for copies of the manuscripts): 

■ A review of the literature review and analysis of Form A data 

■ An analysis of the single subject literature and Form B data 



■ An analysis of the group quantitative literature and Form B data 

■ An analysis of the qualitative literature and Form B data 

■ An analysis of the field validation study results 

■ An analysis of the interdisciplinary literature and practices 
Distribution of Products 

As outlined in our proposal, we sought a commercial publisher in order to ensure that the 
products would be available to a widespread audience and long after the termination of the 
project. Sopris West Publishers (Longmont, CO) agreed to publish all products produced on the 
project. All the products are carried through their catalogue and are marketed by both Sopris 
West and DEC. DEC has marketed the products through NAEYC and CEC who are carrying 
them in their catalogues as well reaching well over 150,000 members of those two organizations 
alone. We have sent free copies of all products to all 619 Coordinators, Part C coordinators, 
OSEP funded early childhood projects, PACER, AAUAP, NECTAS, the Head Start Bureau and 
DEC state affiliates. The products as well as free checklists and various forms of the literature 
that was reviewed are found on the DEC web site: www.dec-sped.org . 

Training/Consultant Bank 

We also engaged in national and regional training and have developed a DEC 
Recommended Practices Consultant Bank which is found on www.dec-sped.org. (see Appendix 
D for graphics of web site, products, trainings and overall activities) The Consultant Bank is 
composed of individuals the strand chairs nominated who are willing to be contacted by local or 
state groups or others for assistance in implementing the practices in the strand of their expertise. 

We have provided training in the following venues: 

National: 
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DEC conference: 1999, 2000, 2001 



NAEYC conference: 2000, 2001 



CEC conference: 1999, 2000, 2002 



National Head Start Association conference: 2001 



CEC Teacher Education Division (TED): 2001 



Regional: 



• New England: June, 2002 

• Washington state: May, 2002 

• Utah: June, 2002 

VIII. GOALS, OBJECTIVES AND ACCOMPLISHMENTS 

The following section outlines the projects goals, objectives and the accomplishments 
related to both. 

Objective One: Synthesize the Knowledge Base 

Activity 1.1 : Identify and classify relevant extant literature: direct services : child 
services, family services, cultural/linguistic competence, technology applications, learning 
environments, interdisciplinary services and assessment; indirect services : policy/procedures, 
personnel development, and systems change (see Appendix A for figure of design). 

Accomplishments: 1) Formulated the problem, operational definitions and the criteria 
with which literature was categorized as relevant and to be included in the review— the 
operational definition for the synthesis was: findings that indicate positive changes in 
development, behavior, learning, functioning, and attitudes attributed to intervention for 
children from birth through five with disabilities, their families, and the personnel who serve 
them. The criteria for the initial inclusion of an article was: any study in a peer referred journal 
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in early intervention or a related field through 1998. 2) The names of journals to be included 
were submitted to the Strand Chairs and Management Team to review and recommend additional 
ones. 3) The journals (48) and studies were selected. 4) Coding sheets were developed for 
generic features and a separate one for each type of research design: single subject, quantitative 
(group), qualitative, mixed method, descriptive/survey (see Appendix B). 5) Coders for all design 
groups were recruited, trained and reliability established. 

The literature review consisted of coders completing the coding sheet for all articles 
which included information about the study (sample, method, results, etc.) as well as the 
practice(s) recommended by the study. This information was entered into a data base. This data 
base can be sorted by article, design, practice and by strand. 

Outcomes: The ten peer reviewed journals originally proposed for the research literature 
synthesis was expanded to include an additional 38 peer reviewed journals. The complete list of 
48 journals utilized in the literature synthesis appears in Table One. This provided a thorough 
review of the literature since 1990; a picture of research focus by strand and by practice, e.g. 
which areas have been more researched than others, etc., and a rich base from which to draw in 
the product development phase of the project. 

Level one screening was completed on all 48 journals resulting in a pool of articles that 
met the broad conceptual definition proposed in the original grant. This initial pool of articles 
was duplicated and filed by journal title in a three-ring binder. 

Level two screening involved the exclusion of certain studies based on their obvious lack of 
relevance to the research synthesis. Articles excluded were removed from binders and filed 
separately. Level two screening resulted in information that made the conceptual and operational 
definitions of which articles to include more apparent. 
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Other procedures and results of the literature are reviewed above. 



Activity 1.2 ; Identify views grounded in experience and values from stakeholders: 

scientific experts, families, practitioners, administrators, and personnel trainers/technical 
assistance providers. 

Accomplishments: Experts from the field were selected as Strand Chairs who: 1) 
assisted in the identification of journals in their area of expertise, 2) held a focus group of other 
scientific experts at the DEC conference in December of 1998, 3) developed a list of 
recommended practices from the focus group to be synthesized with that strand of practices from 
the literature review and other stakeholder groups’ recommendations. Focus groups were also 
conducted at the 1998 DEC conference for three other stakeholder groups: parents, practitioners, 
and administrators to generate lists of recommended practices from their experiences and values 
to be synthesized with those of the scientific experts’ recommendations and the literature. 

Outcomes: 9 strands were selected initially: child services (Dr. Mark Wolery); services to 
the family (Drs. Carl Dunst & Carol Trivette); interdisciplinary services (Dr. Robin McWilliam); 
learning environments (Dr. Mary Beth Bruder); cultural/linguistic competence (Drs. Amy Santos 
& Daphne Thomas); technology (Ms. Kathleen Stremmel); personnel standards (Drs. Vicki 
Stayton & Pat Miller); policy (Dr. Gloria Harbin); systems change/leadership (Dr. Chris 
Salisbury); after the focus groups were held, another was recommended and added: assessment 
(Drs. John Neisworth & Steve Bagnato). All focus groups were conducted and lists of 
recommended practices submitted from all expert strands and the three other stakeholder groups; 
all lists were edited for consistency in format (see Appendix E). 

Grant personnel funded at LSU Health Sciences Center with the assistance of DEC'S 
Family Consortium Chair, conducted a focus group with family members during the DEC 



Annual Meeting in Chicago in December, 1998. Seven family members provided consumer 
perspectives on preferred and valued early intervention/early childhood special education 
practices. Personnel from the University of Wisconsin-Milwaukee conducted a focus group of 
administrators also during the DEC conference. This group of five individuals provided the 
views on recommended practices as experienced by state and local early childhood 
administrators. Personnel from the University of Washington conducted the practitioners focus 
group at the DEC conference. This group of seven made recommendations for practice from the 
experience and knowledge of the early intervention/early childhood special education 
practitioner. The focus groups were audio-taped and either a verbatim transcript or detailed notes 
were generated. A list of recommended practices was derived and sent to focus group 
participants as a member check on data trustworthiness. The three focus group lists of practices 
are in Appendix E. 

Activity 1.3 : Synthesize (syncretize) the extant literature and subjective views: 
(conceptual and integrative review, meta analysis, best evidence, expert opinion). 

Accomplishments: The stakeholder and expert focus group practices were synthesized 
by the Investigators by strand. Each of the three investigators (Smith, McLean and Sandall) 
synthesized the recommendations of the focus groups in three to four strands, then each 
Investigator reviewed each of the lists synthesized by the other Investigators. If there was 
disagreement, the three Investigators discussed the item and reached agreement. These 
synthesized lists were then be reviewed by the strand chairs for validation. The synthesis of these 
focus group practices and the literature practices were completed by March, 2000. 

Outcomes: A synthesis of expert and stakeholder focus group practices has been 
completed and has resulted in the products and training outlined above. 
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Activity 1.4: Identify essential recommended practices (Q methodology, DEC & 



NAEYC review, and expert review). 

Accomplishments: As noted in the earlier narrative, under the advice of OSEP, the 
methodology was redesigned to maintain the complete literature review and product development 
plans. Q studies and DEC and NAEYC review were determined by the Management Team as not 
essential for the integrity of the project outcomes and were not included in this phase. DEC and 
NAEYC leaders were included in the national field validation study. 

Activity 1.5: Identify gaps in research base on recommended practices. 

Accomplishments: A data base has been established of the extant literature. This data 
base can be sorted by strand and by practices. Several analyses of the data base has been 
conducted and five related manuscripts have been written and submitted to peer reviewed 
journals for publication (see above and Appendix C for manuscripts). These manuscripts outline 
areas for further study. 

Outcomes: see Appendix C for the analyses of the data base and recommendations for 
future efforts. 

Objective Two: Determine Product Uses and Effective Formats 

Activity 2.1 : Create a list of recommended practices. 

Accomplishments: As noted above, lists of recommended practices were generated from 
the extant literature and four focus groups. These lists were synthesized and field validated 
resulting in the DEC Recommended Practices in Early Intervention/Early Childhood Special 
Education. 

Outcomes: As noted above, several published books, guides and videos have been 
produced, as well as web site materials including checklists for administrators, parents, and a 
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consultant bank. Training has taken place at 10 national conferences and for three regional 
conferences. 

Activity 2.2: Conduct a field validation of the list of recommended practices 
(internal DEC constituent validation; external constituent validation). 

Accomplishments: The final list of recommended practices were validated by a national 
survey as outlined above. 

Activity 2.3: Conduct focus groups of stakeholders on preferred formats, (families, 
practitioners/administrators, personnel trainers/T.A. providers). 

Accomplishments: Focus groups have been conducted with parents, administrators, 
practitioners and personnel trainers. Each focus group consisted of approximately 1 0 participants 
and asked the same questions. The questions to the groups included: use of current DEC 
recommended practices books and preferred formats for particular uses. 

Outcomes: The focus group recommendations for formats have been compiled and 
reviewed by the members of the groups. These format recommendations were summarized and 
guided the product development activities. 

Activity 2.4: Develop recommended practices formats. 

Accomplishments: Products were developed that met the format suggestions from the 
focus groups and available resources (see above for product descriptions). 

Outcomes: As noted above, a text book, a program assessment guide, a personnel 
development guide and a video were produces and will be carried by a commercial publisher. 
Check lists, literature review materials and a consultant bank are on the DEC web site: www.dec- 
sped.org 

Finally, six manuscripts have been submitted for publication in peer reviewed journals. 
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Activity 2.5; Finalize first drafts of product formats. 



Accomplishments: All products have been drafted and produced except for the personnel 
development guide which is in production. 

Activity 2.6: Field test product formats via case studies. 

Accomplishments: The only product that received a field test was the program 
assessment guide because of time limitations. 

Activity 2.7: Revise product formats. 

Accomplishments: The program assessment guide was revised based on a field test. 
Activity 2.8 : Disseminate products and information about project findings 
Accomplishments: Results and project information have been disseminated at national 
and regional conferences; through six manuscripts submitted to peer reviewed journals, through 
newsletters and brochures and through our national dissemination partners: PACER, AAUAP, 
NECTAS, NAEYC, NHSA, and the Head Start Bureau; and through national mailings of product 
flyers and in some cases of all the products (Part C Coordinators, 619 Coordinators and EC 
Project Directors). See above for review of dissemination activities. 

Objective 3: Link the knowledge base on recommended practices to service delivery. 
Activity 3.1: Develop hi-tech instructional materials (video, CD ROM). 
Accomplishments: based on the suggestions from stakeholder groups, we did not 
develop CD ROM but we did develop a video and web based products. 

Outcomes: one video was produced on child focused practices and is published by Sopris 

West. 

Activity 3.2: Provide educational sessions using a variety of instructional formats. 
Accomplishments: see above for description of conference sessions 
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Activity 3.3: Publish and disseminate final recommended practices in multiple 



product formats. See above 

Activity 3.4: Build systems change capacity. 

Accomplishments: we worked with national dissemination partners to ensure widespread 
knowledge of the recommended practices, we conducted national trainings and regional 
workshops, we distributed hundreds of free copies of all products nation-wide, and we submitted 
a grant proposal to OSEP for a model demonstration project to study systems change and 
adoption of the recommended practices. 

Outcomes: over 200,000 people have either received free copies of the products or have 
received information about the practices and the products. We have received an OSEP grant to 
study the adoption of the recommended practices and systems change strategies in two local 
programs — the Bridging the Gap project, which is in its second year. This project is providing 
not only the opportunity to study the adoption of the practices, but also the opportunity to 
continue the dissemination activities related to the recommended practices including products 
and training. 
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ORIGINAL PROJECT TIMELINES 





Proposed Years/Quarters 


Activities 


1 


2 


3 




1 


2 


3 


4 


1 


2 


3 


4 


1 


2 


OBJECTIVE 1: Synthesize the knowledge base 




Activity 1.1: Identify and classify relevant extant literature: 
direct services: child services, family services, 
cultural/linguistic competence, technology applications, 
learning environments, and interdisciplinary services; 
indirect services: policy/procedures, personnel development, 
and systems change. 


X 


X 


















Activity 1.2: Identify views grounded in experience and 
values from stakeholders: families, practitioners, 
administrators and personnel trainers/T.A. providers. 


X 


X 


















Activity 1.3: Synthesize (syncretize) the extant literature 
and subjective views (conceptual and integrative review, 
meta analysis, best evidence, expert opinion). 




X 


X 


X 














Activity 1.4: Identify essential recommended practices (Q 
methodology, DEC & NAEYC review, and expert review. 








X 
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OBJECTIVE 2: Determine product uses and effective 
formats 




Activity 2.1: Create a list of recommended practices. 








X 


X 












Activity 2.2: Conduct a field validation of the list of 
recommended practices (internal DEC constituent 
validation; external constituent validation). 










X 












Activity 2.3: Conduct focus groups of stakeholders on 
preferred formats, (families, practitioners/administrators, 
personnel trainers/T.A. providers). 






X 


X 














Activity 2.4: Develop recommended practices formats. 












X 










Activity 2.5: Finalize first drafts of product formats. 












X 










Activity 2.6: Field test product formats via case studies. 












X 


X 








Activity 2.7: Revise product formats. 














X 


X 






Activity 2.8: Disseminate products and information about 
the project findings. 






X 


X 


X 


X 


X 


X 


X 


X 


OBJECTIVE 3: Link the knowledge base on recommended 
practices to service delivery. 
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Activity 3.1: Develop hi-tech instructional materials (video, 
CD ROM). 


















X 


X 


Activity 3.2: Provide educational sessions using a variety of 
instructional formats. 
















X 


X 


X 


Activity 3.3: Publish and disseminate final recommended 
practices in multiple product formats. 
















X 


X 


X 


Activity 3.4: Build systems change capacity. 
















X 


X 


X 
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Table 1 



Number of Articles in Each Peer-Reviewed Journal Included in DEC Research Synthesis 



Journal Number of 

Articles 

American Journal of Occupational Therapy 32 

American Journal on Mental Retardation 50 

Archives of Physical Medicine and Rehabilitation 2 

Augmentative and Alternative Communication 1 5 

Behavior Modification 1 1 

Behavior Therapy 4 

Behavioral Disorders 1 

Child Development 6 

Child: Care, Health and Development 13 

Developmental Medicine and Child Neurology 40 

Developmental Psychology 5 

Diagnostique 6 

Early Childhood Research Quarterly 1 1 

Early Education and Development 21 

Education and Training in Mental Retardation and Developmental 1 8 

Disabilities 

Education and Treatment of Children 1 6 
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Exceptional Children 47 

Exceptionality 3 

Family Relations 13 

Infants and Y oung Children 1 2 

Infant-Toddler Intervention 45 

Journal of Abnormal Child Psychology 8 

Journal of Applied Behavior Analysis 55 

Journal of Applied Developmental Psychology 1 3 

Journal of Behavioral Education 1 1 

Journal of Communication Disorders 1 

Journal of Early Intervention 1 02 

Journal of Emotional and Behavioral Disorders 7 

Journal of Marriage and Family 0 

Journal of Pediatric Nursing 1 3 

Journal of Pediatric Psychology 7 

Journal of Special Education 5 

Journal of Special Education Technology 0 

Journal of Speech, Language, and Hearing Research 26 

Journal of The Association for Persons with Severe Handicaps 32 

Journal of Visual Impairment & Blindness 1 8 

Language, Speech and Hearing Services in Schools 18 

Mental Retardation 22 
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Merrill-Palmer Quarterly 



0 



Neonatal Network 9 

Occupational Therapy Journal of Research 6 

Pediatric Physical Therapy 9 

Pediatrics 7 

Physical & Occupational Therapy in Pediatrics 12 

Physical Therapy 2 

Teacher Education and Special Education 1 

Topics in Early Childhood Special Education 73 

Volta Review 7 



Note: Journals in italics designate those that contributed 20 or more articles to the review. 
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Table 2 



Number and Percent of Articles Interpreted By Coders as Having at Least One Recommended 
Practice bv Method 



Method 


N (%) 


Group Quantitative 


450 (54%) 


Single Subject 


184 (22%) 


Descriptive / Survey 


121 (15%) 


Qualitative 


74 (9%) 


Mixed Method * 


6(1%) 


Total 


835 (100%) 



* Mixed method = some combination of group quantitative, qualitative, descriptive / survey and 
single subject methods 
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Number and Percent of Articles bv Research Method Employed and Recommended Practices Strand Placement 
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Table 4 



Number of Articles bv Method and Publication Year 



Method 


Year 




1990 


1991 


1992 


1993 


1994 


1995 


1996 


1997 


1998 


Group 


41 


43 


40 


60 


62 


51 


53 


67 


33 


Quantitative 


54.7% 


51.8% 


48.8% 


53.1% 


55.9% 


52.6% 


55.2% 


66.3% 


42.9% 




14 


13 


23 


33 


27 


21 


20 


13 


20 


Single Subject 


18.7% 


15.7% 


28.1% 


29.2% 


24.3% 


21.7% 


20.8% 


12.9% 


26.0% 


Descriptive / 


14 


22 


14 


16 


15 


13 


10 


6 


11 


Survey 


18.7% 


26.5% 


17.1% 


14.2% 


13.5% 


13.4% 


10.4% 


5.9% 


14.3% 




6 


5 


4 


4 


7 


9 


13 


14 


12 


Qualitative 


8.0% 


6.0% 


4.9% 


3.5% 


6.3% 


9.3% 


13.5% 


13.9% 


15.6% 


Mixed 


0 


0 


1 


0 


0 


3 


0 


1 


1 


Methods * 






1.2% 






3.1% 




1.0% 


1.3% 


Total 


75 


83 


82 


113 


111 


97 


96 


101 


77 



* Mixed method = some combination of group quantitative, qualitative, descriptive/survey and 
single subject methods 
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Table 5 



Number and Percent of Articles Reporting Disability Categories of Child Study Participants by 
Year 



Disability 










Year 










Category 






















1990 


1991 


1992 


1993 


1994 


1995 


1996 


1997 


1998 




6 


15 


10 


8 


15 


9 


8 


13 


15 


Autism 


5.1% 


12.8% 


7.7% 


4.9% 


8.9% 


6.1% 


5.6% 


7.3% 


15.3% 




14 


8 


13 


16 


13 


16 


12 


16 


6 


Cerebral Palsy 


11.9% 


6.8% 


10.0% 


9.9% 


7.7% 


10.9% 


8.4% 


9.0% 


6.1% 


Developmental 


15 


15 


20 


36 


23 


27 


24 


31 


22 


Delay 


12.7% 


12.8% 


15.4% 


22 . 2 % 


13.7% 


18.4% 


16.8% 


17.4% 


22.4% 


Developmental 


5 


7 


9 


10 


9 


8 


10 


11 


8 


Disability 


4.2% 


6.0% 


6.9% 


6.2% 


5.4% 


5.4% 


7.0% 


6.2% 


8.2% 


Down 


9 


13 


14 


15 


21 


20 


16 


16 


7 


Syndrome 


7.6% 


11.1% 


10.8% 


9.3% 


12.5% 


13.6% 


11.2% 


9.0% 


7.1% 


Emotional/ 


4 


5 


5 


6 


9 


2 


5 


10 


4 


Behavioral 


3.4% 


4.3% 


3.8% 


3.7% 


5.4% 


1.4% 


3.5% 


5.6% 


4.1% 


Disorder 




















Hearing 


5 


2 


6 


11 


8 


7 


10 


14 


5 


Impairment/ 


4.2% 


1.7% 


4.6% 


6.8% 


4.8% 


4.8% 


7.0% 


7.9% 


5.1% 


Deaf 























Mental 


18 


15 


14 


15 


16 


13 


16 


17 


8 


Retardation 


15.3% 


12.8% 


10.8% 


9.3% 


9.5% 


8.8% 


11.2% 


9.6% 


8.2% 


Speech/ 


9 


3 


10 


11 


16 


11 


12 


14 


4 


Language 


7.6% 


2.6% 


7.7% 


6.8% 


9.5% 


7.5% 


8.4% 


7.9% 


4.1% 


Impairment 




















Visual 


5 


9 


7 


6 


6 


5 


5 


10 


4 


Impairment/ 


4.2% 


7.7% 


5.4% 


3.7% 


3.6% 


3.4% 


3.5% 


5.6% 


4.1% 


Blindness 






















28 


24 


22 


27 


32 


29 


25 


26 


15 


Other 


23.7% 


20.5% 


16.9% 


16.7% 


19.0% 


19.7% 


17.5% 


14.6% 


15.3% 
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Table 6 



Number and Percent of Articles Reporting Severity of Disability by Year 





Year 




1990 


1991 


1992 


1993 


1994 


1995 


1996 


1997 


1998 


Articles 


55 


51 


48 


71 


66 


51 


61 


54 


42 


Reporting 


73% 


61% 


59% 


63% 


59% 


53% 


64% 


53% 


55% 


Severity 
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Table 7 



Number and Percent of Articles Reporting Study Setting Data 





Number / Percent of 


Study Setting 


Articles 




225 


Home 


23.4% 




230 


Hospital/Clinic 


23.9% 




170 


Inclusive/Integrated Classroom 


17.7% 




130 


Separate or Segregated Classroom 


13.5% 




33 


Experimental Settings/Separate from peers 


3.4% 




152 


Other - Survey/Interview 


15.8% 




23 


Other - Not clear 


2.4% 


Totals 


963 
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Table 8 



Number and Percent of Articles Reporting Family Income and Education Levels bv Year 













Year 












1990 


1991 


1992 


1993 


1994 


1995 


1996 


1997 


1998 


Articles 




















Reporting 


16 


16 


19 


29 


22 


19 


23 


22 


18 


Family Income 
Level 


21% 


19% 


23% 


26% 


20% 


20% 


24% 


22% 


23% 



Articles 



Reporting 

Family 

Education 



20 


18 


22 


29 


22 


25 


34 


34 


26 


27% 


22% 


27% 


26% 


20% 


26% 


35% 


33% 


34% 



Level 



50 

o 

ERIC 



45 



References 



Casto, G., & Mastropieri, M. A. (1986). The efficacy of early intervention programs: A 
meta-analysis. Exceptional Children, 52, 417-424. 

Casto, G., White, K., & Taylor, C. (1983). Final report 1982-1983 work scope. Logan, 
UT: Early Intervention Research Institute, Utah State University. 

Division for Early Childhood (DEC) (2001). DEC recommended practices video: 
Selected strategies for teaching young children with special needs [video]. Longmont, CO: 
Sopris West. 

Division for Early Childhood (DEC) Task Force on Recommended Practices (Eds.). 
(1993). DEC recommended practices: Indicators of quality in programs for infants and young 
children with special needs and their families. Reston, VA: Council for Exceptional Children. 

Dunst, C. J., & Snyder, S. W. (1986). A critique of the Utah State University early 
intervention meta-analysis research. Exceptional Children, 53(3), 269-276. 

Garland, C. W. (1995). DEC recommended practices: Who uses them? Journal of Early 
Intervention, 19, 18-20. 

Greenbaum, T.L. (1987). The practical handbook and guide to focus group research. 
Lexington, MA: Lexington Press. 

Guralnick, M. (1997). The effectiveness of early intervention. Baltimore: Paul Brookes 
Publishing Co. 

Hemmeter, M.L., Joseph, G.E., Smith, B.J., & Sandall, S. (2001). DEC recommended 
practices program assessment: Improving practices for young children with special needs and 
their families. Longmont, CO: Sopris West. 



51 

o 

ERIC 



46 



Meisels, S. J., & Shonkoff, J. P. (Eds.). (1990). Handbook of early childhood 
intervention. Cambridge, England: Cambridge University Press. 

Odom, S. L., & McLean, M. E. (1996). Early intervention/early childhood special 
education: Recommended practices. Austin, TX: Pro-Ed Publishing Co. 

Sandall, S., McLean, M. E., & Smith, B. J. (Eds.). (2000). DEC recommended practices 
in early intervention/ early childhood special education. Longmont, CO: Sopris West. 

Stayton, V.D, Miller, P.S., & Dinnebeil, L.A. (in press). Personnel preparation in early 
childhood special education: Implementing the DEC recommended practices. Longmont, CO: 
Sopris West. 

Strain, P. S., & Smith, B. J. (1986). A counter-interpretation of early intervention effects: 
A response to Casto and Mastoprieri. Exceptional Children, 53, 260-265. 

Tjossem, T. (1976). Intervention strategies for high risk infants and young children. 
Baltimore: University Park Press. 

White, K., Mastropieri, M., & Casto, G. (1984). An analysis of special education early 
childhood projects approved by the joint dissemination Review Panel. Journal of the Division for 
Early Childhood, 9,1 1-26^ 

Wolery, M. (1995). Some concerns about process. Journal of Early Intervention, 19, 21- 
23. 



52 

ERIC 



47 



Appendix A 

Figures 



o 

ERIC 

hffliflaffHEaoaa 



53 



DEC RECOMMENDED PRACTICES 
STRAND CHAIRS 



Direct Services (b-5) 

* Child Services - 

* Family Services - 

* Cultural & Linguistic Competence - 

* Technological Applications - 

* Learning Environments - 

* Interdisciplinary Models - 

* Assessment - 

Indirect Services/Supports 

* Policy & Procedures - 

* Personnel Development - 

* Systems Change, Maintenance & 

Leadership Climate - 

Strand Eiaison/Coordinator 

* Dr. Sam Odom 



Dr. Mark Wolery 

Drs. Carl Dunst and Carol Trivette 

Drs. Amy Santos and Daphne Thomas 

Ms. Kathleen Stremel 

Dr. Mary Beth Bruder 

Dr. Robin Me William 

Drs. John Neisworth and Steve Bagnato 

Dr. Gloria Harbin 

Drs. Vicki Stayton and Pat Miller 

Dr. Chris Salisbury 
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DEC RECOMMENDED PRACTICES PROJECT: 

Ten Subcategories of Professional Practices by Two Early Childhood Age Groups 
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Figure 1. Project Design (continued) 
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Figure 1. Data collection, analysis, and synthesis of recommendations from data sources 
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Appendix B 

Literature Review 
Coding Forms 



Name of Coder 



Date Sent 

Date Received 
(Rev. 5/31/99) 



PART A: GENERIC CODING SHEET 



1. Study ID Number 

2. Year of Publication 

3. Source 

SAMPLE INFORMATION 

4. Study participants (check only one) 

□ 01. Children with disabilities - Total number 

□ 02. Children without disabilities - Total number 

□ 03. Families of children with disabilities - Total number 

□ 04. Families of children without disabilities - Total number 

□ 05. Personnel - Total Number 

Please specify roles and numbers of each 



□ 06. Combination of the above (please specify type of participant and numbers of each) 



5 . Gender of Participants 

□ 01. Male; N = % = 

□ 02. Female; N = % = 

□ 03. Not specified 

6. Study participants chronological age range(s) 

7. Mean age(s) of study participants 

8. Standard deviation(s) of study participant(sD) ages if reported 

9. Race of Participants (check all that apply) 



□ 01. 


African-American; 


N = % = 


□ 02. 


Asian- American; 


N = % = 


□ 03. 


Latino- American ; 


N = % = 


□ 04. 


White; 


N = % = 


□ 05. 


Native American; 


N = % = 


□ 06. 


Other (please specify race 


and number or percentage) 



□ 07. Not specified 
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Name of Coder 

10. Disability of Participants (check all that apply) 



□ 


01 


Cerebral Palsy; 


N = 


% 


□ 


02. 


Down Syndrome; 


N = 


% 


□ 


03. 


Mental Retardation; 


N = 


% 


□ 


04. 


Developmental Disability; 


N = 


% 


□ 


05. 


Autism; 


N = 


% 


□ 


06. 


Visual impairment/blindness; 


N = 


% 


□ 


07. 


Hearing impairment/deaf; 


N = 


% 


□ 


08. 


Speech/Language impairment; 


N = 


% 


□ 


09. 


Emotional/Behavioral Disorder; 


N = 


% 


□ 


10. 


Developmental Delay; 


N = 


% 


□ 


11. 


Other (please specify); 


N = 


% 



11. Is the severity of disabilities or developmental levels for children discussed? 

□ 01. Yes 

□ 02. No 

1 2. If yes, what levels are identified? For example: mild to severe; mental age; developmental quotient? 



13. Is the educational level of any family member reported? 

□ 01. Yes 

□ 02. No 

If yes, list reported descriptive statistics. For example: mean, ranges, categories, etc. 



1 4. Are any family income levels reported? 

□ 01. Yes 

□ 02. No 

15. If yes, list reported descriptive statistics, for example, mean, ranges, categories, etc. 



16. Study Setting: (check all that apply) 



□ 


01. 


Home 


□ 


02. 


Segregated classroom 


□ 


03. 


Inclusive or integrated classroom 


□ 


04. 


Hospital/clinic 


□ 


05. 


Other (please specify) 
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Name of Coder 



(Rev. 5/31/99) 



PART B: SPECIALIZED CODING SHEET FOR 
QUALITATIVE STUDIES 

1 Includes a description of the conceptual framework. 

□ 01. Yes 

□ 02. No 

2. What is the general focus of the study? . 



3. Number of study participants. 



4. 



5. 



6. 



Type of study. 

□ 01. Ethnographic 

□ 02. Naturalistic 

□ 03. Interpretive 

□ 04. Case study 

□ 05. Participant observation 

□ 06. Phenomenological 

□ 07. Historical 

□ 08. Other (please specify) 

Includes a description of the procedures. 

□ 01. Participant selection 

□ 02. Data collection methods 

Data source(s). (Check all that apply) 

□ 01. Observation 

□ 02. Interviews 

□ 03. Documents 

□ 04. Combination of data sources (please describe) 

□ 05. Other (please specify) 



7. What was the length of the data collection period? 



8. Includes description of data analysis. (Check all that apply) 

□ 01. Detailed 

□ 02. Appropriate 

□ 03. Inductive 



9. 




Evidence of credibility. (Check all that apply) 

□ 01. Multiple methods 

□ 02. Member checks 

□ 03 Audit trail 

□ 04. Triangulation across methods 

□ 05. Triangulation across sources 

□ 06. Intercoder reliability checks 

□ 07. Other (please specify) 



Name of Coder 



10. Includes a discussion of the meaning of the findings in terms of theory. 

□ 01. Yes 

□ 02. No 



RECOMMENDED PRACTICES 

Please write all recommended practice(s) supported in the study. Practices may be 
supported by the data even if the authorsO conclusions do not explicitly acknowledge 
them. Practices may be related to contextual variables in addition to, or instead of, the 
variable(s) examined in the study. Recommended practices are written in the present 
tense, using active voice, with people as the subject of the sentence. Examples of 
specified practices gleaned from qualitative literature for use as models follow. 



Exemplars 

1) Program staff including therapists should explain to families the advantages and 
disadvantages of different therapy delivery models (e.g., home versus clinic-based). 

2) Program staff including therapists should explain to families the pros and cons related to 
frequency of the delivery of therapy services, (e.g., once per week, once per month, not 
at all). 

3) Professionals should use skills that facilitate the communication process, such as 
listening skills, openness to suggestions, responsivity and a willingness to disclose 
information to promote collaboration with families. 

1 1 . Recommended Practice #1 . 



12. Check the most appropriate strand placement for the above identified recommended 
practice. 

Recommended Practice #1 



□ 01. Child-focused 

□ 02. Family-centered 

□ 03. Cultural/linguistic competency 



63 



Name of Coder 

□ 04. Technology application 

□ 05. Learning environments 

□ 06. Interdisciplinary models 

□ 07. Policy and procedure 

□ 08. Personnel development 

□ 09. Systems change, maintenance and leadership climate 

□ 10. Assessment 

□ 11. Other (please specify) 

13. Recommended Practice #2. 



■*\ 



14. Check the most appropriate strand placement for the above identified recommended 
practice. 

Recommended Practice #2 

□ 01. Child-focused 

□ 02. Family-centered 

□ 03. Cultural/linguistic competency 

□ 04. Technology application 

□ 05. Learning environments 

□ 06. Interdisciplinary models 

□ 07. Policy and procedure 

□ 08. Personnel development 

□ 09. Systems change, maintenance and leadership climate 

□ 10. Assessment 

□ 11. Other (please specify) 

15. Recommended Practice #3. 



16. Check the most appropriate strand placement for the above identified recommended 
practice. 
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Name of Coder 

Recommended Practice #3 



□ 


01. 


Child-focused 


□ 


02. 


Family-centered 


□ 


03. 


Cultural/linguistic competency 


□ 


04. 


Technology application 


□ 


05. 


Learning environments 


□ 


06. 


Interdisciplinary models 


□ 


07. 


Policy and procedure 


□ 


08. 


Personnel development 


□ 


09. 


Systems change, maintenance and leadership climate 


□ 


10. 


Assessment 


□ 


11. 


Other (please specify) 



1 7. Recommended Practice #4. 



18. Check the most appropriate strand placement for the above identified recommended 
practice. 

Recommended Practice #4 



□ 


01. 


Child-focused 


□ 


02. 


Family-centered 


□ 


03. 


Cultural/linguistic competency 


□ 


04. 


Technology application 


□ 


05. 


Learning environments 


□ 


06. 


Interdisciplinary models 


□ 


07. 


Policy and procedure 


□ 


08. 


Personnel development 


□ 


09. 


Systems change, maintenance and leadership climate 


□ 


10. 


Assessment 


□ 


11. 


Other (please specify) 



If you have identified additional empirically validated practices, write them and the most 
appropriate strand placement on the back of this page or on a continuation page. 



O 
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Name of Coder (Rev. 7/12/99) 

PART B: SPECIALIZED CODING SHEET FOR 
GROUP DESIGN QUANTITATIVE STUDIES 

A. Sampling - Sampling refers to processes associated with participant selection and 
assignment to groups 

1. Were participants randomly selected from a defined population? □ 01. Yes □ 02. No 

2. Were participants randomly assigned to groups? □ 01. Yes □ 02. No 

3. Were participants matched on any attribute variables? □ 01. Yes □ 02. No 

4. If yes, list (e g., developmental age) 



5. Was a control group (i.e., group that received no treatment) used? □ 01. Yes □ 02. No 



6 . 

7. 



Was a covariate(s) used? 

If yes, please list the covariate(s). 



□ 01. Yes □ 02. No 



B. Independent Variable^ and Associated Categories - Independent variables encompass 
variables actively manipulated by the researcher (i.e., experimental variables) and attribute 
variables (i.e., assigning participants to groups based on an attribute such as “severity of 
disability”), which are examined as part of substantive hypothesis(es) 



List, or briefly describe, the independent variable(s) and the categories associated with the 
variable(s): 



IV #1: 



Categories associated with IV #1: 



IV #2 (if applicable): 



Categories associated with IV # 2 : 



O 
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Name of Coder 

IV #3 (if applicable): 



Categories associated with IV #3 : 



(if applicable, list other IVs and associated categories on back of page) 

If the study is pre-experimental (i.e., everyone receives the same treatment or is assigned to a 
single category of treatment) or causal comparative (i.e, “independent variable” being examined is 
an attribute or ability variable that cannot be manipulated by the researcher), briefly describe the 
“treatment” experienced by all participants: 



C. Fidelity of Treatment 

8. Did the author(s) provide information related to fidelity of treatment implementation? 

□ 01. Yes □ 02. No 

9. If yes, briefly describe. 



D. Duration of Study 

10a. Information provided about length of intervention or treatment received? □ 01. Yes □ No. 
10b. If yes, check all that apply. 

□ 01. Length in hours/minutes 

□ 02. Length in days 

□ 03. Length in months 

□ 04. Frequency of sessions 

Length of sessions (e.g., minutes, hours, if specified) 

□ 05. Other duration data 



E. Dependent Vari ablets) and Dependent Measure! si 

List, or briefly describe, the dependent variable(s) and the dependent measures used to define the 
variable(s): 

DV #1: 
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Name of Coder 

Dependent Measure #1 : 



DV #2 (if applicable): 



Dependent Measure #2 (if applicable): 



DV #3 (if applicable): 



Dependent Measure #3 (if applicable): 



(if applicable, list other DVs and associated dependent measures on back of page) 

1 1 . Was information provided about reliability/agreement of study measures? 

□ 01. Yes □ 02. No 

12. If yes, what type(s) of reliability/agreement data were provided? (check all that apply) 

□ 01. Test-retest 

□ 02. Internal consistency (e.g., Cronbach’s alpha) 

□ 03. Parallel forms 

□ 04. Interrater reliability 

□ 05. Standard error of measurement 

□ 06. Interobserver agreement 

13. Were reported reliability coefficients based on data obtained from the study sample? 

□ 01. Yes □ 02. No □ 03. Not Applicable 

14. Was information provided about the validity of study measures? □ 01. Yes □ 02. No 



Name of Coder 

15. If yes, what type(s) of validity data were provided? (check all that apply) 

□ 01. Face/content 

□ 02. Predictive validity 

□ 03. Concurrent validity 

□ 04. Construct (e.g., factor analytic evidence) 

□ 05. Other (specify) 

16. Were reported validity coefficients based on data obtained from the study sample? 

□ 01. Yes □ 02. No □ 03 . Not Applicable 

17. For dependent measures involving observational data collection, were measures of 
interobserver agreement or reliability calculated? □ 01. Yes □ 02. No 

□ 03 . Not Applicable 

18. If yes, what type(s) of agreement or reliability index was provided? (check all that apply) 

□ 01. Simple agreement 

□ 02. Kappa 

□ 03. Weighted kappa 

□ 04. Intraclass correlation coefficient (ICC) 

□ 05. Other (specify) 

F. Major Study Hypothesises! 

List the major substantive hypothesis(es) evaluated by the author(s): 

Hypothesis # 1 



Hypothesis #2 (if applicable): 



Hypothesis #3 (if applicable): 



Name of Coder 

Hypothesis #4 (if applicable): 



(if applicable, list other hypotheses on the back of page) 

G. Statistical Analyses 

19. Which of the following types of inferential statistics were used to test the hypotheses listed 
above? (check all that apply) 

□ 01. Univariate parametric statistics (e.g., t-tests, ANOVA, multiple regression, 

Pearson r) 

□ 02. Univariate nonparametric statistics (e.g., Spearman rho, Mann Whitney U Test, 

chi square contingency table, Friedman 2-Way ANOVA, Wilcoxon matched pairs 
sign rank test) 

□ 03. Multivariate parametric statistics (e.g., MANOV A, discriminant analysis, 

canonical correlation analysis) 

□ 04. Multivariate nonparametric statistics (e.g., logistic regression) 

□ 05. Other (e.g.. Structural Equation Modeling [SEM], Confirmatory Factor Analysis, 

Hierarchical Linear Models [HLM)]), please specify 

H. Effect Sizefsl for Major Study Hvpothesis(es) and Empirically Validated Prac tice(s): 

20. For major study hypothesis #1 is an effect size measure reported? 

□ 01. Yes □ 02. No 

21. If yes, check type of measure reported and list value. 

□ 01 . eta squared (D 2 ) = 

□ 02. omega squared (D 2 ) = 

□ 03. multivariate lambda (□, e.g., Wilks’ lambda) = 

□ 04. bivariate correlation ( r or r 2 ) = 

□ 05. coefficient of determination (R or R 2 ) = 

□ 06. Adjusted R 2 (Wherry) = 

□ 07. Cohen’s d(d) = 

□ 08. Cohen’s k ( k ) = 

□ 09. Glass’ delta (A) = 

□ 10. Phi squared (D 2 ) = 75 

□ 11. other (please specify) 

□ 12. effect size reported, but type of measure not specified = 

o If the anal y sis for ma i or stud y hypothesis #1 was factorial (e.g., 2-Way ANOVA), were any 

! . Slf. interactions statistically significant? □ 01. Yes □ 02. No 
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23. If yes, briefly describe the nature of the interactions (e.g., the treatment was more effective 
for children with more significant disabilities) 



24. For major study hypothesis #2 is an effect size measure reported? 

□ 01. Yes □ 02. No 

25. If yes, check type of measure reported and list value. 

□ 01. eta squared (CD 2 ) = 

□ 02. omega squared (ID 2 ) = 

□ 03. multivariate lambda (□, e.g., Wilks’ lambda) = 

□ 04. bivariate correlation (r or r 2 ) = 

□ 05. coefficient of determination ( R or Z? 2 ) = 

□ 06. Adjusted R 2 (Wherry) = 

□ 07. Cohen’s d (cf) = 

□ 08. Cohen’s k ( k ) = 

□ 09. Glass’ delta (A) = 

□ 10. Phi squared (CD 2 ) = 

□ 11. other (please specify) 

□ 12. effect size reported, but type of measure not specified = 

26. If the analysis for major study hypothesis #2 was factorial (e.g., 2-Way ANOVA), were any 
interaction effects involving the empirically validated practice statistically significant? 

01. Yes □ 02. No 

27. If yes, briefly describe the nature of the interaction (e.g., the treatment was more effective 
for children with more significant disabilities) 



28. For major study hypothesis #3 is an effect size measure reported? 
□ 01. Yes □ 02. No 
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29. If yes, check type of measure reported and list value. 

□ 01. eta squared (0 2 ) = 

□ 02. omega squared (0 2 ) = 

□ 03. multivariate lambda (□, e g., Wilks’ lambda) = 

□ 04. bivariate correlation (r or r 2 ) = 

□ 05. coefficient of determination (Z? or Z? 2 ) = 

□ 06. Adjusted Z? 2 (Wherry) = 

□ 07. Cohen’s d (d) = 

□ 08. Cohen’s k ( k ) = 

□ 09. Glass’ delta (A) = 

□ 10. Phi squared (0 2 ) = 

□ 11. other (please specify) 

0 12. effect size reported, but type of measure not specified = 

30. If the analysis for major study hypothesis #3 was factorial (e.g., 2-Way ANOVA), were any 
interaction effects involving the empirically validated practice statistically significant? 

□ 01. Yes □ 02. No 

31. If yes, briefly describe the nature of the interaction (e.g., the treatment was more effective 

for children with more significant disabilities) 



32. For major study hypothesis #4 is an effect size measure reported? 

□ 01. Yes □ 02. No 

33. If yes, check type of measure reported and list value. 

□ 01. eta squared (0 2 ) = 

□ 02. omega squared (0 2 ) = 

□ 03. multivariate lambda (□, e.g., Wilks’ lambda) = 

□ 04. bivariate correlation ( r or r 2 ) = 

□ 05. coefficient of determination ( R or Z? 2 ) = 

□ 06. Adjusted Z? 2 (Wherry) = 

□ 07. Cohen’s d (d) = 

□ 08. Cohen’s k ( k ) = 

□ 09. Glass’ delta (A) = 

□ 10. Phi squared (0 2 ) = 

□ 11. other (please specify) 

□ 12. effect size reported, but type of measure not specified = _ 
O 
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34. If the analysis for major study hypothesis #4 was factorial (e g., 2-Way ANOVA), were any 
interaction effects involving the empirically validated practice statistically significant? 

□ 01. Yes □ 02. No 

35. If yes, briefly describe the nature of the interaction (e.g., the treatment was more effective 
for children with more significant disabilities) 



If additional hypotheses are listed above, put answers to the questions related to effect size 
and factorial results on the back of this page. 
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RECOMMENDED PRACTICES 

Please write all recommended practice(s) supported in the study. Practices may be supported by 
the data even if the authors’ conclusions do not explicitly acknowledge them. Practices may be 
related to contextual variables in addition to, or instead of, the variable(s) examined in the study. 
Recommended practices are written in the present tense, using active voice, with people as the 
subject of the sentence. An example of a specified practice gleaned from quantitative literature 
for use as a model follows. 

Exemplar 

Adults provide opportunities for young children with hearing impairments to play with 
peers who are not hearing impaired to increase play complexity. 

Recommended Practice #1. 



Check the most appropriate strand placement for the above identified recommended practice. 

Recommended Practice #1 

01. Child-focused 

02. Family-centered 

03. Cultural/linguistic competency 

04. Technology application 

05. Learning environments 

06. Interdisciplinary models 

07. Policy and procedure 

08. Personnel development 

09. Systems change, maintenance and leadership climate 
10. Assessment 

1 1 . Other (please specify) _ 

Recommended Practice #2. 
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Check the most appropriate strand placement for the above identified recommended practice. 

Recommended Practice #2 

01. Child-focused 

02. Family-centered 

03. Cultural/linguistic competency 

04. Technology application 

05. Learning environments 

06. Interdisciplinary models 

07. Policy and procedure 

08. Personnel development 

"• Systems change, maintenance and leadership climate 

10. Assessment 

11. Other (please specify) 

Recommended Practice #3. 



Check the most appropriate strand placement for the above identified recommended practice. 

Recommended Practice #3 

01. Child-focused 

02. Family-centered 

03. Cultural/linguistic competency 

04. Technology application 

05. Learning environments 

06. Interdisciplinary models 

07. Policy and procedure 

08. Personnel development 

09 Systems change, maintenance and leadership climate 

10. Assessment 

11. Other (please specify) 

Recommended Practice #4. 



8U 
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Check the most appropriate strand placement for the above identified recommended practice. 
Recommended Practice #4 



01 . 

02 . 

03 . 

04 . 

05 . 

06 . 

07 . 

08 . 

09 . 

10 . 
11 . 



Child-focused 
Family-centered 
Cultural/linguistic competency 
Technology application 
Learning environments 
Interdisciplinary models 
Policy and procedure 
Personnel development 

Systems change, maintenance and leadership climate 
Assessment 

Other (please specify) 



If additional empirically validated practices are listed above, write them and the most 
appropriate strand placement on the back of this page or on a continuation page. 
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PART B: SPECIALIZED CODING SHEET FOR SINGLE SUBJECT STUDIES 

1 . Design Type 

□ 01. Withdrawal of treatment 

□ 02. Reversal 

□ 03. Multiple baseline across subjects, settings, etc. 

□ 04. Multiple probe 

□ 05. Simultaneous treatment 

□ 06. Alternating treatment 

□ 07. Other 

2. Interpersonal Ratio 

□ 01. Pull-out individual/separate 

□ 02. Group/integrated 

□ 03. Other 

□ 04. Not specified 

3. Action Agent 

□ 01. Adult-guided 

□ 02. Peer-mediated 

□ 03. Other 

4. Number of Replications (e g., multiple baseline across three settings=3 replications; 
ABAB with 4 subjects=8 replications 



5 . Describe Dependent Variable Ultimately Plotted (e g., % intervals of engagement; rate of 

tantrums; % of corrects/errors) 



6. Describe Independent Variable 



O 
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Independent Variable Assessment 

□ 01. Yes 

□ 02. No 
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8. If yes, fidelity of intervention was: 

□ 01. High 

□ 02. Medium 

□ 03. Low 

9. Evidence of Improvement Over Time 
Within Phase(s) 

□ 01. Yes 

□ 02. No 

10. Across Phase(s) 

□ 01. Yes 

□ 02. No 

1 1 . Social Validity Assessments Included 
Intervention Acceptability 

□ 01. Yes 

□ 02. No 

12. Outcome Acceptability/ Acknowledgment 

□ 01. Yes 

□ 02. No 

1 3 . Evidence of Intervention Maintenance Across Time 

□ 01. Yes 

□ 02. No 

14. If yes, length of follow-up 

15. Evidence of Intervention Generalization 

□ 01. Yes 

□ 02. No 

16. If yes, degree of match with intervention performance: 



□ 


01. 


High 


□ 


02. 


Medium 


□ 


03. 


Low 



RECOMMENDED PRACTICES 

Please write all recommended practice(s) supported in the study. Practices may be 
supported by the data even if the authorsD conclusions do not explicitly acknowledge 
them. Practices may be related to contextual variables in addition to, or instead of, the 
variable(s) examined in the study. Recommended practices are written in the present 
tense, using active voice, with people as the subject of the sentence. Examples of 
specified practices gleaned from the singe subject literature for use as models follows. 
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Exemplars 

1) Teachers embed instructional opportunities into routine class transitions to increase 
learning outcomes for children with mild developmental disabilities. 

2) Practitioners teach mothers of preschoolers with language impairments to use effectively 
a variety of milieu teaching strategies (e.g, model, mand-model, time delay, and 
incidental) in home settings for increasing the amount and complexity of childrenOs 
language production. 

1 7. Recommended Practice # 1 



18. Check the suggested strand placement for the above identified recommended practice. 
Recommended Practice #1 

□ 01. Child-focused 

□ 02. Family-centered 

□ 03. Cultural/linguistic competency 

□ 04. Technology application 

□ 05. Learning environments 

□ 06. Interdisciplinary models 

□ 07. Policy and procedure 

□ 08. Personnel development 

□ 09. Systems change, maintenance and leadership climate 

□ 10. Assessment 

□ 11. Other (please specify) 

19. Recommended Practice #2. 
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20. Check the suggested strand placement for the above identified recommended practice. 
Recommended Practice #2 

□ 01. Child-focused 

□ 02. Family-centered 

□ 03. Cultural/linguistic competency 

□ 04. Technology application 

□ 05. Learning environments 

□ 06. Interdisciplinary models 

□ 07. Policy and procedure 

□ 08. Personnel development 

□ 09. Systems change, maintenance and leadership climate 

□ 10. Assessment 

□ 11. Other (please specify) 

21. Recommended Practice #3. 



♦ 



22. Check the suggested strand placement for the above identified recommended practice. 
Recommended Practice #3 

□ 01. Child-focused 

□ 02. Family-centered 

□ 03. Cultural/linguistic competency 

□ 04. Technology application 

□ 05. Learning environments 

□ 06. Interdisciplinary models 

□ 07. Policy and procedure 

□ 08. Personnel development 

□ 09. Systems change, maintenance and leadership climate 

□ 10. Assessment 

□ 11. Other (please specify) 

23. Recommended Practice #4. 



O 
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24 Check the suggested strand placement for the above identified recommended practice. 



Recommended Practice #4 



□ 


01. 


□ 


02. 


□ 


03. 


□ 


04. 


□ 


05. 


□ 


06. 


□ 


07. 


□ 


08. 


□ 


09. 


□ 


10. 


□ 


11. 



Child-focused 
Family-centered 
Cultural/linguistic competency 
Technology application 
Learning environments 
Interdisciplinary models 
Policy and procedure 
Personnel development 

Systems change, maintenance and leadership climate 
Assessment 

Other (please specify) 



If you have identified additional empirically validated practices, write them and the most 
appropriate strand placement on the back of this page or on a continuation page. 
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(Rev. 5/31/99) 



PART B: SPECIALIZED CODING SHEET 
SHORT-FORM FOR SURVEY/DESCRIPTIVE 



1. Were participants randomly selected from a defined population? □ 01. Yes □ 02. No 

2. Were participants randomly assigned to groups? □ 01. Yes □ 02. No 

3. Were participants matched on any attribute variables? □ 01. Yes □ 02. No 

4. If yes, list (e g., developmental age) 

5. Was information provided about reliability/agreement of study measures?01. Yes □ 02. No 

6. If yes, what type(s) of reliability/agreement data were provided? (check all that apply) 

□ 01. Test-retest 

□ 02. Internal consistency (e.g., Cronbach’s alpha) 

□ 03 . Parallel forms 

□ 04. Interrater 

□ 05. Standard error of measurement 

□ 06. Interob server agreement 

7. Were reported reliability coefficients based on data obtained from the study sample? 

□ 01. Yes □ 02. No □ 03. Not Applicable 

8. Was information provided about the validity of study measures? □ 01. Yes □ 02. No 

9. If yes, what type(s) of validity data were provided? (check all that apply) 

□ 01. Face/content 

□ 02. Predictive validity 

□ Concurrent validity 

□ Construct (e.g., factor analytic evidence) 

□ 05. Other (specify) 

10. If yes, were reported validity coefficients based on data obtained from the study sample? 

□ 01. Yes □ 02. NoD 03 . Not Applicable 
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RECOMMENDED PRACTICES 

Please write all recommended practice(s) supported in the study. Practices may be supported by 
the data even if the authors’ conclusions do not explicitly acknowledge them. Practices may be 
related to contextual variables in addition to, or instead of, the variable(s) examined in the study. 
Recommended practices are written in the present tense, using active voice, with people as the 
subject of the sentence. An example of a specified practice gleaned from quantitative literature 
for use as a model follows. 

Exemplar 

Adults should provide opportunities for young children with hearing impairments to play 
with peers who are not hearing impaired to increase play complexity. 

Recommended Practice #1. 



Check the most appropriate strand placement for the above identified recommended practice. 

Recommended Practice #1 

01. Child-focused 

02. Family-centered 

03. Cultural/linguistic competency 

04. Technology application 

05. Learning environments 

06. Interdisciplinary models 

07. Policy and procedure 

08. Personnel development 

09. Systems change, maintenance and leadership climate 

10. Assessment 

1 1 . Other (please specify) 

Recommended Practice #2. 



O 

ERIC 



88 



Name of Coder 



Check the most appropriate strand placement for the above identified recommended practice. 



Recommended Practice #2 

01. Child-focused 

02. Family-centered 

03. Cultural/linguistic competency 

04. Technology application 

05. Learning environments 

06. Interdisciplinary models 

07. Policy and procedure 

08. Personnel development 

09. Systems change, maintenance and leadership climate 
10. Assessment 

1 1 . Other (please specify) 

Recommended Practice #3. 



Check the most appropriate strand placement for the above identified recommended practice. 
Recommended Practice #3 



01 . 

02. 

03. 

04. 

05. 

06. 

07. 

08. 

09. 

10 . 
11 . 



Child-focused 
Family-centered 
Cultural/linguistic competency 
Technology application 
Learning environments 
Interdisciplinary models 
Policy and procedure 
Personnel development 

Systems change, maintenance and leadership climate 
Assessment 

Other (please specify) 



O 
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Recommended Practice #4. 



Check the most appropriate strand placement for the above identified recommended practice 

Recommended Practice #4 

01. Child-focused 

02. Family-centered 

03. Cultural/linguistic competency 

04. Technology application 

05. Learning environments 

06. Interdisciplinary models 

07. Policy and procedure 

08. Personnel development 

09. Systems change, maintenance and leadership climate 

10. Assessment 

1 1 . Other (please specify) 



If additional empirically validated practices are listed above, write them and the most 
appropriate strand placement on the back of this page or on a continuation page. 
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(Rev. 7/12/99) 



A. 

1 . 

2 . 

3. 

4. 

5. 

6. 
7. 



SPECIALIZED CODING SHEET FOR MIXED METHOD STUDIES 
SECTION I (QUANTITATIVE) 



Sampling - Sampling refers to processes associated with participant selecti on and 
assignment to groups 



Were participants randomly selected from a defined population? 
Were participants randomly assigned to groups? 

Were participants matched on any attribute variables? 



□ 01. Yes □ 02. No 

□ 01. Yes □ 02. No 

□ 01. Yes □ 02. No 



If yes, list (e g., developmental age) 



Was a control group (i.e., group that received no treatment) used? □ 01. Yes □ 

Was a covariate(s) used? ^ es ^ 

If yes, please list the covariate(s) 



02. No 
02. No 



B Independent Variable(s) and Associated Categories - Independent variables encompass 
variables actively manipulated by the researcher (i.e., experimental variables) and attribute 
variables (i.e., assigning participants to groups based on an attribute such as “seventy of 
disability”), which are examined as part of substantive hypothesis(es) 

List, or briefly describe, the independent variable(s) and the categories associated with the 
variable(s): 

IV #1: 



Categories associated with IV #1: 



IV #2 (if applicable): 



Categories associated with IV #2: 



O 
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IV #3 (if applicable): 



Categories associated with IV #3: 



(if applicable, list other IVs and associated categories on back of page) 

If the study is pre-experimental (i.e., everyone receives the same treatment or is assigned to a 
single category of treatment) or causal comparative (i.e, “independent variable” being examined is 
an attribute or ability variable that cannot be manipulated by the researcher), briefly describe the 
“treatment” experienced by all participants: 



C. Fidelity of Treatment 

8. Did the author(s) provide information related to fidelity of treatment implementation? 
□ 01. Yes □ 02. No 

9. If yes, briefly describe. 



E. Duration of Study 

10a. Information provided about length of intervention or treatment received? □ 01. Yes □ No. 
10b. If yes, check all that apply. 

□ 01. Length in hours/minutes 

□ 02. Length in days 

□ 03. Length in months 

□ 04. Frequency of sessions 

□ 05. Length of sessions (e.g., minutes, hours, if specified) 

□ 06. Other duration data 



E. Dependent Van ablet si and Dependent Measure(sl 

List, or briefly describe, the dependent variable(s) and the dependent measures used to define the 
variable(s): 

DV #1: 
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Dependent Measure #1: 



DV #2 (if applicable): 



Dependent Measure #2 (if applicable): 



DV #3 (if applicable): 



Dependent Measure #3 (if applicable): 



(if applicable, list other DVs and associated dependent measures on back of page) 

1 1 Was information provided about reliability/agreement of study measures? 

□ 01. Yes □ 02. No 

12. If yes, what type(s) of reliability/agreement data were provided? (check all that apply) 

□ 01. Test-retest 

□ 02. Internal consistency (e g., Cronbach’s alpha) 

□ 03. Parallel forms 

□ 04. Interrater reliability 

□ 05. Standard error of measurement 

□ 06. Interobserver agreement 

1 3 . Were reported reliability coefficients based on data obtained from the study sample? 

□ 01. Yes □ 02. No □ 03. Not Applicable 



14. Was information provided about the validity of study measures? 



□ 01. Yes □ 02. No 
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15. If yes, what type(s) of validity data were provided? (check all that apply) 

□ 01. Face/content 

□ 02. Predictive validity 

□ 03 . Concurrent validity 

□ 04. Construct (e g., factor analytic evidence) 

□ 05. Other (specify) 

16. Were reported validity coefficients based on data obtained from the study sample? 

□ 01. Yes □ 02. No □ 03 . Not Applicable 

17. For dependent measures involving observational data collection, were measures of 
interobserver agreement or reliability calculated? □ 01. Yes □ 02. No 

□ 03 . Not Applicable 

18. If yes, what type(s) of agreement or reliability index was provided? (check all that apply) 

□ 01. Simple agreement 

□ 02. Kappa 

□ 03. Weighted kappa 

□ 04. Intraclass correlation coefficient (ICC) 

□ 05. Other (specify) 

F. Major Study Hypothesises! 

List the major substantive hypothesis(es) evaluated by the author(s): 

Hypothesis #1 



Hypothesis #2 (if applicable): 



Hypothesis #3 (if applicable): 
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Hypothesis #4 (if applicable): 



(if applicable, list other hypotheses on the back of page) 

G. Statistical Analyses 

19. Which of the following types of inferential statistics were used to test the hypotheses listed 
above? (check all that apply) 

□ oi. Univariate parametric statistics (e.g., t-tests, ANOVA, multiple regression, 

Pearson r) 

□ 02. Univariate nonparametric statistics (e.g., Spearman rho, Mann Whitney U Test, 

chi square contingency table, Friedman 2-Way ANOVA, Wilcoxon matched pairs 
sign rank test) 

□ 03. Multivariate parametric statistics (e.g., MANOVA, discriminant analysis, 

canonical correlation analysis) 

□ 04. Multivariate nonparametric statistics (e.g., logistic regression) 

□ 05. Other (e.g.. Structural Equation Modeling [SEM], Confirmatory Factor Analysis, 

Hierarchical Linear Models [HLM)]), please specify 

H. Effect Sizefs^ for Major Study Hvpothesis(esl and Empirically Va lidated Practice(s): 

20. For major study hypothesis #1 is an effect size measure reported? 

□ 01. Yes □ 02. No 

21. If yes, check type of measure reported and list value. 

□ 01. eta squared (D 2 ) = 

□ 02. omega squared (D 2 ) = 

□ 03. multivariate lambda (□, e.g., Wilks’ lambda) = 

□ 04. bivariate correlation (r or r 2 ) = 

□ 05. coefficient of determination ( R or R 2 ) = 

□ 06. Adjusted R 2 (Wherry) = . 

□ 07. Cohen’s d (d) = 

□ 08. Cohen’s k ( k ) = 

□ 09. Glass’ delta (A) = 

□ 10. Phi squared (D 2 ) = 

□ 11. other (please specify) 

□ 12. effect size reported, but type of measure not specified = 

"9 If the analysis for major study hypothesis #1 was factorial (e.g., 2-Way ANOVA), were any 

ERIC interactions statistically significant? □ OLYes □ 02. No 



Name of Coder 

23. If yes, briefly describe the nature of the interactions (e g., the treatment was more effective 
for children with more significant disabilities) 



24. For major study hypothesis #2 is an effect size measure reported? 

□ 01. Yes □ 02. No 

25. If yes, check type of measure reported and list value. 

□ 01. eta squared (D 2 ) = 

□ 02. omega squared (D 2 ) = 

□ 03. multivariate lambda (□, e.g., Wilks’ lambda) = 

□ 04. bivariate correlation (r or r 2 ) = 

□ 05. coefficient of determination ( R or R 2 ) = 

□ 06. Adjusted R 2 (Wherry) = 

□ 07. Cohen’s d (d) = 

□ 08. Cohen’s k (k) = 

□ 09. Glass’ delta (A) = 

□ 10. Phi squared (D 2 ) = 

□ 11. other (please specify) 

□ 12. effect size reported, but type of measure not specified = 

26. If the analysis for major study hypothesis #2 was factorial (e.g., 2-Way ANOVA), were any 

interaction effects involving the empirically validated practice statistically significant? 

□ 01. Yes □ 02. No 

27. If yes, briefly describe the nature of the interaction (e.g., the treatment was more effective 

for children with more significant disabilities) 




28. 



For major study hypothesis #3 is an effect size measure reported? 
□ 01. Yes □ 02. No 
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29 If yes, check type of measure reported and list value. 

□ 01 . eta squared (D 2 ) = 

□ 02. omega squared (□ ) = 

□ 03. multivariate lambda (□, e g., Wilks’ lambda) = 

□ 04. bivariate correlation (r or r 2 ) = 

□ 05. coefficient of determination ( R or R 2 ) = 

□ 06. Adjusted R 2 (Wherry) = 

□ 07. Cohen’s d (d) = 

□ 08. Cohen’s k ( k ) = 

□ 09. Glass’ delta (A) = 

□ 10. Phi squared (D 2 ) = 

□ 11. other (please specify) 

□ 12. effect size reported, but type of measure not specified = 

30. If the analysis for major study hypothesis #3 was factorial (e.g., 2-Way ANOVA), were any 
interaction effects involving the empirically validated practice statistically significant? 

□ 01. Yes □ 02. No 

3 1 If yes, briefly describe the nature of the interaction (e.g., the treatment was more effective 
for children with more significant disabilities) 



32. For major study hypothesis #4 is an effect size measure reported? 
□ 01. Yes □ 02. No 



33 if yes, check type of measure reported and list value. 




□ 01. eta squared (D 2 ) = 

□ 02. omega squared (D 2 ) = 

□ 03. multivariate lambda (□, e.g., Wilks’ lambda) = 

□ 04. bivariate correlation ( r or r 2 ) = 

□ 05. coefficient of determination ( R or R 2 ) = 

□ 06. Adjusted tf 2 (Wherry) = . 

□ 07. Cohen’s d (d) = 

□ 08. Cohen’s k ( k ) = 

□ 09. Glass’ delta (A) = 

□ 10. Phi squared (D 2 ) = 

□ 11. other (please specify) 

□ 12. effect size reported, but type of measure not specified = 

97 
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34. If the analysis for major study hypothesis #4 was factorial (e.g., 2-Way ANOVA), were any 
interaction effects involving the empirically validated practice statistically significant? 

□ 01. Yes □ 02. No 

35. If yes, briefly describe the nature of the interaction (eg., the treatment was more effective 

for children with more significant disabilities) 



If additional hypotheses are listed above, put answers to the questions related to effect size 
and factorial results on the back of this page. 
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RECOMMENDED PRACTICES 

Please write all recommended practice(s) supported in the study. Practices may be supported by 
the data even if the authors’ conclusions do not explicitly acknowledge them. Practices may be 
related to contextual variables in addition to, or instead of, the variable(s) examined in the study. 
Recommended practices are written in the present tense, using active voice, with people as the 
subject of the sentence. An example of a specified practice gleaned from quantitative literature 

for use as a model follows. 

Exemplar 

Adults provide opportunities for young children with hearing impairments to play with 
peers who are not hearing impaired to increase ploy complexity. 

Recommended Practice 



Check the most appropriate strand placement for the above identified recommended practice. 
Recommended Practice #1 

01. Child-focused 

02. Family-centered 

03. Cultural/linguistic competency 

04. Technology application 

05. Learning environments 

06. Interdisciplinary models 

07. Policy and procedure 

08. Personnel development 

09. Systems change, maintenance and leadership climate 
10. Assessment 

11. Other (please specify) 



Recommended Practice #2. 
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Check the most appropriate strand placement for the above identified recommended practice. 

Recommended Practice #2 

01. Child-focused 

02. Family-centered 

03. Cultural/linguistic competency 

04. Technology application 

05. Learning environments 

06. Interdisciplinary models 

07. Policy and procedure 

08. Personnel development 

09. Systems change, maintenance and leadership climate 

10. Assessment 

1 1 . Other (please specify) 

Recommended Practice #3. 



Check the most appropriate strand placement for the above identified recommended practice. 

Recommended Practice #3 

01. Child-focused 

02. Family-centered 

03. Cultural/linguistic competency 

04. Technology application 

05. Learning environments 

06. Interdisciplinary models 

07. Policy and procedure 

08. Personnel development 

09. Systems change, maintenance and leadership climate 

10. Assessment 

1 1 . Other (please specify) 






Recommended Practice #4. 



Name of Coder 



Check the most appropriate strand placement for the above identified recommended practice. 
Recommended Practice #4 

01. Child-focused 

02. Family-centered 

03. Cultural/linguistic competency 

04. Technology application 

05. Learning environments 

06. Interdisciplinary models 

07. Policy and procedure 

08. Personnel development 

09. Systems change, maintenance and leadership climate 
10. Assessment 

11. Other (please specify) 

If additional empirically validated practices are listed above, write them and the most 
appropriate strand placement on the back of this page or on a continuation page. 
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(Rev. 5/31/99) 



1 . 

2. 

3. 

4. 



5. 

6. 



7. 

8 . 

9. 



O 




SPECIALIZED CODING SHEET FOR MIXED METHOD STUDIES 
SECTION II (QUALITATIVE) 

Includes a description of the conceptual framework. 

□ 01. Yes 

□ 02. No 

What is the general focus of the study? 



Number of study participants. 

Type of study. 

□ 01. Ethnographic 

□ 02. Naturalistic 

□ 03. Interpretive 

□ 04. Case study 

□ 05. Participant observation 

□ 06. Phenomenological 

□ 07. Historical 

□ 08. Other (please specify) 

Includes a description of the procedures. 

□ 01. Participant selection 

□ 02. Data collection methods 

Data source(s). (Check all that apply) 

□ 01. Observation 

□ 02. Interviews 

□ 03. Documents 

□ 04. Combination of data sources (please describe) _ 

□ 05. Other (please specify) 

What was the length of the data collection period? 

Includes description of data analysis. (Check all that apply) 

□ 01. Detailed 

□ 02. Appropriate 

□ 03. Inductive 

Evidence of credibility. (Check all that apply) 

□ 01. Multiple methods 

□ 02. Member checks 

□ 03. Audit trail 

□ 04. Triangulation across methods 

□ 05. Triangulation across sources 

□ 06. Intercoder reliability checks 

□ 07. Other (please specify) 




Name of Coder 



Includes a discussion of the meaning of the findings in terms of theory. 

□ 01. Yes 

□ 02. No 

RECOMMENDED PRACTICES 

Please write all recommended practice(s) supported in the study. Practices may be supported by the 
data even if the authorsD conclusions do not explicitly acknowledge them. Practices may be related 
to contextual variables in addition to, or instead of, the variable(s) examined in the study. 
Recommended practices are written in the present tense, using active voice, with people as the 
subject of the sentence. Examples of specified practices gleaned from qualitative literature for use as 

models follow. 



Exemplars 

1) Program staff including therapists should explain to families the advantages and disadvantages 
of different therapy delivery models (e.g., home versus clinic-based). 

2) Program staff including therapists should explain to families the pros and cons related to 
frequency of the delivery of therapy services, (e.g., once per week, once per month, not at all). 

3) Professionals should use skills that facilitate the communication process, such as listening skills, 
openness to suggestions, responsivity and a willingness to disclose information to promote 
collaboration with families. 

1 1 . Recommended Practice # 1 



Check the most appropriate strand placement for the above identified recommended practice. 



Recommended Practice #1 




□ 


01. 


□ 


02. 


□ 


03. 


□ 


04. 


□ 


05. 


□ 


06. 


□ 


07. 


□ 


08. 


□ 


09. 


□ 


10. 


a 


11. 



Child-focused 
F amily-centered 
Cultural/linguistic competency 
Technology application 
Learning environments 
Interdisciplinary models 
Policy and procedure 
Personnel development 

Systems change, maintenance and leadership climate 

Assessment 103 

Other (please specify) — 



Name of Coder 

1 3 . Recommended Practice #2. 



14. Check the most appropriate strand placement for the above identified recommended practice. 
Recommended Practice #2 



□ 


01. 


Child-focused 


□ 


02. 


Family-centered 


□ 


03. 


Cultural/linguistic competency 


□ 


04. 


Technology application 


□ 


05. 


Learning environments 


□ 


06. 


Interdisciplinary models 


□ 


07. 


Policy and procedure 


□ 


08. 


Personnel development 


□ 


09. 


Systems change, maintenance and leadership climate 


□ 


10. 


Assessment 


□ 


11. 


Other (please specify) 



1 5 . Recommended Practice #3 . 



16. Check the most appropriate strand placement for the above identified recommended practice. 
Recommended Practice #3 



□ 


01. 


Child-focused 


□ 


02. 


Family-centered 


□ 


03. 


Cultural/linguistic competency 


□ 


04. 


Technology application 


□ 


05. 


Learning environments 


□ 


06. 


Interdisciplinary models 


□ 


07. 


Policy and procedure 


□ 


08. 


Personnel development 


□ 


09. 


Systems change, maintenance and leadership climate 


□ 


10. 


Assessment 


□ 


11. 


Other (please specify) 
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Name of Coder 



17. Recommended Practice #4. 



18. Check the most appropriate strand placement for the above identified recommended practice. 



Recommended 


Practice #4 


□ 01. 


Child-focused 


□ 02. 


Family-centered 


□ 03. 


Cultural/linguistic competency 


□ 04. 


Technology application 


□ 05. 


Learning environments 


□ 06. 


Interdisciplinaiy models 


□ 07. 


Policy and procedure 


□ 08. 


Personnel development 


□ 09. 


Systems change, maintenance and leadership climate 


□ 10. 


Assessment 


□ 11. 


Other (please specify) 



If you have identified additional empirically validated practices, write them and the most appropriate 
strand placement on the back of this page or on a continuation page. 
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Name of Coder _ (Rev. 5/12/99) 

SPECIALIZED CODING SHEET FOR MIXED METHOD STUDIES 
SECTION HI (SINGLE SUBJECT) 



1 . Design Type 

□ 01. Withdrawal of treatment 

□ 02. Reversal 

□ 03. Multiple baseline across subjects, settings, etc. 

□ 04. Multiple probe 

□ 05. Simultaneous treatment 

□ 06. Alternating treatment 

□ 07. Other 

2. Interpersonal Ratio 

□ 01. Pull-out individual/separate 

□ 02. Group/integrated 

□ 03. Other 

□ 04. Not specified 

3 . Action Agent 

□ 01. Adult-guided 

□ 02. Peer-mediated 

□ 03. Other 

4. Number of Replications (e.g., multiple baseline across three settings=3 replications; 
ABAB with 4 subjects=8 replications 



5. Describe Dependent Variable Ultimately Plotted (e.g., % intervals of engagement; rate of 
tantrums; % of corrects/errors) 



6. Describe Independent Variable 



7 




Independent Variable Assessment 

□ 01. Yes 

□ 02. No 
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Name of Coder 

8. If yes, fidelity of intervention was: 

□ 01. High 

□ 02. Medium 

□ 03. Low 

9. Evidence of Improvement Over Time 

Within Phase(s) 

□ 01. Yes 

□ 02. No 

10. Across Phase(s) 

□ 01 Yes 

□ 02. No 

1 1 . Social Validity Assessments Included 
Intervention Acceptability 

□ 01. Yes 

□ 02. No 

1 2 . Outcome Acceptability/ Acknowledgment 

□ 01. Yes 

□ 02. No 

1 3 . Evidence of Intervention Maintenance Across Time 

□ 01. Yes 

□ 02. No 

14. If yes, length of follow-up 

1 5 . Evidence of Intervention Generalization 

□ 01. Yes 

□ 02. No 

16. If yes, degree of match with intervention performance: 

□ 01. High 

□ 02. Medium 

□ 03. Low 

RECOMMENDED PRACTICES 

Please write all recommended practice(s) supported in the study. Practices may be 
supported by the data even if the authorsD conclusions do not explicitly acknowledge 
them. Practices may be related to contextual variables in addition to, or instead of, the 
variable(s) examined in the study. Recommended practices are written in the present 
tense, using active voice, with people as the subject of the sentence. Examples of 
O specified practices gleaned from the singe subject literature for use as models follows. 

ERJC in 7 



Name of Coder 

Exemplars 

1) Teachers embed instructional opportunities into routine class transitions to increase 
learning outcomes for children with mild developmental disabilities. 

2) Practitioners teach mothers of preschoolers with language impairments to use effectively 
a variety of milieu teaching strategies (e.g., model, mand-model, time delay, and 
incidental) in home settings for increasing the amount and complexity of childrenOs 
language production. 

1 7 . Recommended Practice # 1 . 



18. Check the suggested strand placement for the above identified recommended practice. 
Recommended Practice #1 



□ 


01. 


Child-focused 


□ 


02. 


Family-centered 


□ 


03. 


Cultural/linguistic competency 


□ 


04. 


Technology application 


□ 


05. 


Learning environments 


□ 


06. 


Interdisciplinary models 


□ 


07. 


Policy and procedure 


□ 


08. 


Personnel development 


□ 


09. 


Systems change, maintenance and leadership climate 


□ 


10. 


Assessment 


□ 


11. 


Other ( please specify) 



19. Recommended Practice #2. 



20. Check the suggested strand placement for the above identified recommended practice. 
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Name of Coder 



Recommended Practice #2 

□ 01. Child-focused 

□ 02. Family-centered 

□ 03. Cultural/linguistic competency 

□ 04. Technology application 

□ 05. Learning environments 

□ 06. Interdisciplinary models 

□ 07. Policy and procedure 

□ 08. Personnel development 

□ 09. Systems change, maintenance and leadership climate 

□ 10. Assessment 

□ 11. Other (please specify) 

21. Recommended Practice #3. 



22. Check the suggested strand placement for the above identified recommended practice. 

Recommended Practice #3 

□ 01. Child-focused 

□ 02. Family-centered 

□ 03. Cultural/linguistic competency 

□ 04. Technology application 

□ 05. Learning environments 

□ 06. Interdisciplinary models 

□ 07. Policy and procedure 

□ 08. Personnel development 

□ 09. Systems change, maintenance and leadership climate 

□ 10. Assessment 

□ 11. Other (please specify) - 

23. Recommended Practice #4. 



24. Check the suggested strand placement for the above identified recommended practice. 



O 
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Name of Coder 

Recommended Practice #4 



□ 


01. 


Child-focused 


□ 


02. 


Family-centered 


□ 


03. 


Cultural/linguistic competency 


□ 


04. 


Technology application 


□ 


05. 


Learning environments 


□ 


06. 


Interdisciplinary models 


□ 


07. 


Policy and procedure 


□ 


08. 


Personnel development 


□ 


09. 


Systems change, maintenance and leadership climate 


□ 


10. 


Assessment 


□ 


11. 


Other (please specify) 



If you have identified additional empirically validated practices, write them and the most 
appropriate strand placement on the back of this page or on a continuation page. 
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Abstract 



Results are reported from an analysis of the research literature involving children 
with disabilities, age birth through five, their families, the personnel who serve them and 
policies and systems change strategies related to the provision of services. The work 
reported in this paper was one part of a national effort by the Division for Early 
Childhood (DEC) of the Council for Exceptional Children (CEC) and several 
participating Universities to develop a set of evidence-based recommended practices for 
the field. Research articles appearing in 48 peer-reviewed journals from 1990 to 1998 
were reviewed. Analyses of the literature review data-base reveal trends in research 
methods used for studying particular topics, as well as information about populations and 
settings that have been studied during this time period. The authors suggest implications 
for future research in EI/ECSE. 
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For over 30 years, literature in early intervention/early childhood special 
education (EI/ECSE) has reported strategies, methods, and tools for improving the 
developmental and educational outcomes for young children with disabilities (Guralnick, 
1997; Meisels & Shonkoff, 1990; Tjossem, 1976; White, Mastropieri, & Casto, 1984). 
Previous efforts to synthesize the EI/ECSE knowledge base have been limited. For 
example, in the 1980s researchers employed meta-analytic techniques to synthesize the 
early intervention efficacy literature (Casto & Mastroprieri, 1986; Casto, White, & 
Taylor, 1983); however, large bodies of relevant research (e.g., single-subject and 
qualitative studies) were not included in the synthesis (Dunst & Snyder, 1986; Strain & 
Smith, 1986). 

Another effort to summarize the EI/ECSE knowledge base, sponsored by the 
Division for Early Childhood (DEC) of the Council for Exceptional Children (CEC), was 
conducted in 1991. Recommended practices in fourteen areas, called “strands”, were 
generated by groups composed of researchers, practitioners, representatives from higher 
education, and families. These practices were subsequently validated by a national 
sample of 500 individuals representing these stakeholder groups. The resulting list of 405 
practices, first published in 1993 in a DEC document (DEC Task Force on 
Recommended Practices, 1993) was later followed by a book (Odom & McLean, 1996) 
that further explained the practices. Although this effort was generally well received, the 
passage of time and limitations associated with the 1991 effort signaled the need for an 
updated set of practices and a more systematic and comprehensive synthesis of research 
and practices in EI/ECSE (Garland, 1995; Wolery, 1995). The 1991 DEC practices were 
derived solely from stakeholder focus groups; no systematic review of the scientific 
literature was conducted to validate the extent to which the recommended practices were 



5 



supported with research evidence. Additionally, the recommendations only addressed 
direct services and personnel development. There were no recommendations for 
administration or policy support necessary for the delivery of quality direct services. 
Finally, the landscape of EI/ECSE has changed significantly since DEC’s 1991 work on 
the original recommended practices project. The number of preschool programs has 
grown dramatically, empirical research has expanded, services are provided in more 
inclusive settings, partnerships with general early childhood education have grown, and 
all states are participating in Part C of the Individuals with Disabilities Education Act 
(IDEA). 

In the fall of 1998, DEC and several collaborating university partners were 
awarded a federal grant to (a) produce a thoroughly supported synthesis of research and 
experienced-based knowledge about EI/ECSE practices for young children with 
disabilities birth through 5 and their families; (b) translate and disseminate the synthesis 
in user-friendly formats for practitioners, families, administrators and training/technical 
assistance providers; and (c) address the administrative and systems change foundations 
necessary for long-term adoption of quality practices. 

More rigorous methods for producing a set of recommended practices were used 
in this effort to increase the likelihood of their adoption and implementation at the child, 
family, and systems levels. These improved methods consisted of: reviewing the research 
literature for practices that result in improved outcomes; conducting focus groups of 
researchers, parents, practitioners, and administrators to ascertain their beliefs and values 
about practices that result in improved outcomes; synthesizing these sources of 
information; producing recommendations for both “direct” services practices (with 
children and families) and “indirect” practices or supports such as personnel preparation 
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and policy and systems change activities necessary for the implementation of 
recommended practices with children and families; and conducting a national field- 
validation of the practices. The methods which produced the latest compilation of DEC 
Recommended Practices in Early Intervention/Early Childhood Special Education 
(Sandall, McLean, & Smith, 2000) are summarized in Figure 1., and are described in 
detail elsewhere (see Smith, McLean, Sandall, Snyder, & Broudy, 2000). The purpose of 
this article is to describe the literature review process and subsequent analyses of the 
research literature data-base emphasizing the research trends between 1990 and 1998. 



The procedures used to compile the recommended practices were accomplished in 
three phases. Phase 1 involved identifying recommended practices through focus groups 
and a comprehensive literature review. In Phase 2, practices developed in the focus 
groups were synthesized with practices generated from the literature review. Phase 3 
involved a field-validation of the synthesized practices. This manuscript focuses on the 
literature review associated with Phase 1 . 

Overall, the project employed a mixed-method design with an emphasis on 
integrative approaches. From the beginning, a management team of eight researchers was 
established to ensure that all decisions were subjected to expert review and represented a 
consensus of opinion. The management team consisted of four project investigators, a 
strand liaison/methodology consultant, and three other methodology consultants 
representing expertise in qualitative, single-subject, and group quantitative designs. 

The management team determined the categories of practice (strands) to be used 
for the purposes of grouping and organizing the information from the literature review. 
The seven strands of recommended practices were: five direct services— assessment, 



Methods and Procedures 
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child-focused practices, family-based practices, interdisciplinary models, technology 
applications; and two indirect supports— personnel preparation, and policies, procedures 
and systems change. 

Research published in peer-reviewed professional journals from 1990 through 
1998 was included in the literature review. Forty-eight journals relevant to the field of 
EI/ECSE were selected and reviewed (see Table 1. for journals and number of articles in 
each). The tables of contents and abstracts of articles in the 48 journals from 1990 to 
1998 were reviewed by project staff to determine if the article met the criteria for 
inclusion in the study. The criteria were that the article (a) must be an original research 
report and (b) involved children with disabilities, birth through 5, families of children 
with disabilities birth through 5, personnel who serve them, or policies and systems 
change procedures that support effective practice with this population. Each article that 
met these criteria was analyzed for major aspects of research design and results. Analysis 
of the articles was accomplished by having individual coders read articles and complete 
two coding forms that were developed by the project investigators and methodology 
consultants. 

A generic coding form (Form A) was used uniformly across all articles. This form 
was designed to record the following information (see Appendix A for copy of form): 

• article title, authors, journal, year 

• general description of participants (children with disabilities, families of 
children with disabilities, etc.) 

• gender, mean age and ethnicity of participants 

• disability of participants and severity level of disability 

• educational level of family members 
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• income level of family members 

• setting in which study was conducted 

A specialized coding sheet (Form B) designed specifically to highlight additional 
information about research rigor based on the methodology used in the article was also 
completed by the coders for each article. Coding forms for group quantitative design, 
single subject design, descriptive/survey design, qualitative design, and mixed method 
design were used. The specialized coding forms were developed by the project’s 
methodology consultants to ensure that each article was appropriately reviewed 
according to the research method used in the article. Example items on the specialized 
coding forms included: variable identification, research design features, outcome 
measures, duration of intervention, findings, recommended practice(s) supported by the 
study, and the strand that the recommended practice reflected (see accompanying articles 
in this issue for analyses of these data). A main purpose of the specialized coding forms 
was to help the coder evaluate the scientific rigor of the article and subsequently lead the 
reviewer to determine recommended practice(s) that was supported by the research 
evidence. 

Forty-two coders were trained to code Form A and one specialized form. Inter- 
coder agreement of at least 85% was established with a methodology consultant before 
coders engaged in formal coding. In a similar effort to conduct a review of evidence- 
based practice in occupational therapy, coding sheets were developed and an inter-coder 
agreement of 75-86% was reached (see Law, Stewart, Pollock, Letts, Bosch, 
Westmorland, & Philpot, 2002). 

Following training, coders were assigned between 10 and 40 articles to code using 
both the generic and appropriate specialized form for each assigned article. Twenty-nine 




119 



9 



additional coders read approximately one third of the articles included in the initial 
screening {n = 360) and independently coded the articles as to the recommended 
practice(s) and strand placement generated by the first coder. When discrepancies 
occurred between the two coders related to the recommended practices or the strand 
placement, the article was reviewed and read by two members of the management team 
and consensus was reached about the recommended practice and strand placement. This 
occurred for less than 5% of the articles. 

Results/Trends 

The number of articles derived from the initial screening was 1,018. The final 
number of articles included in the review was 835. Articles (183) were excluded by 
coders if the research conducted did not lead directly to a recommended practice, the 
article did not meet the original criteria for inclusion in the study, or a coder and a 
methodology consultant determined that the article did not meet the projects’ scientific 
standards to merit inclusion (e.g., case studies had been incorrectly determined to be 
single subject design studies). Of the 835 articles coded, 54% (n=450) employed a group 
quantitative design, 22% (n=184) used a single subject design, 15% (n=121) used 
descriptive/survey designs, 9% (n= 74) qualitative designs, and 1% (w=6) used mixed 
methods. Table 2 shows the final number of articles with research findings interpreted by 
coders as having implications for recommended practices by research method. 

The total number of recommended practices derived from the 835 articles 
reviewed was 1,453. This total number of recommended practices exceeds the number of 
articles reviewed because according to the coders, some studies generated evidence that 
produced more than one recommended practice. Table 3 shows the number and percent 
of articles according to the research methodology employed and the recommended 
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practice strand assigned by the coder. For example, 399 articles supported the Child- 
Focused strand. One hundred and eighty-seven (47%) of those articles used a group 
quantitative method, 158 (40%) used a single subject design, 38 (10%) a survey/ 
descriptive design, 12 (3%) qualitative approaches and 4(1%) used mixed-method 
desiga Literature support for practices were noted 977 times across the seven strands and 
across the five different methodologies. This number is higher than the actual number of 
articles (835) because some articles contained more than one practice and at the same 
time supported two or more strands in the judgment of the coders. Further review of 
Table 3 shows that: (a) a disparity exists in the amount of empirical research that supports 
the various practice strands and, (b) regardless of practice strand, group quantitative 
methods were the predominant research method employed during 1990-1998 in this 
EI/ECSE research. 

Table 4 displays the study methods employed by articles by year. For instance, 
group quantitative methods were used in 41 (54.7%) of the 1990 articles, while single 
subject and descriptive/survey designs were used in 14 (18.7%), qualitative in 6 (8.0%) 
and mixed methods were used in no articles that year. There is a general trend towards 
increased numbers of articles employing qualitative methodology over the 9 years. The 
use of single subject methodology increased from 1990 to 1998 but varied with each year. 
Group quantitative methodology remained the highest percentage method used across all 
years. 

Tables 5 through 8 depict analyses of the Form A data. Table 5 shows the number 
and percent of articles that reported the disability category of child study participants by 
year. The “other” category included coder comments such as: “at risk for developmental 
disability”, neurologically impaired”, “fetal alcohol syndrome”, “spina bifida”, “sick 
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preemies”. The coders were instructed to code the disability category as defined by the 
article. Therefore, there were categories or subcategories that were not on the coding 
sheet. The coders coded these under “other” rather than make an interpretation. 

Table 6 shows the number and percent of articles reporting the level of severity of 
disability of child study participants by year. These data show that on average 53%-73% 
of articles report the severity of disability with the percent declining over time from 73% 
in 1990 to 55% in 1998. 

Table 7 shows the number and percent of articles reporting the study setting. For 
the study setting variable, no explicit definitions were offered to coders. Instructions were 
to code what the author(s) reported as the setting of the study. The setting options for 
coders on Form A were: home, hospital/clinic, inclusive/integrated classroom, segregated 
classroom, or other. Initially the “other” category contained nearly 40% of the responses. 

Two project staff reviewed the entries in the “other” category and agreed to (a) 
distinguish “survey/interview” (15.8%), and “not clear” entries (2.4%); (b) add 
“separate” to the segregated classroom category for entries that described settings that are 
segregated but not a classroom (“speech lab”, “OT room”, etc.); (c) create a new category 
of “experimental settings/separate from peers” for entries that described the study setting 
as “experimental setting”, “quiet room”, etc.; and, (d) add entries related to “community” 
to the “inclusive/integrated” category. Table 7 shows that nearly half of the research 

settings in this EI/ECSE literature between 1990-1998 were home or hospital/clinic. Less 

\ 

than 20% of studies were conducted in inclusive/integrated classrooms or community 
settings. 

Table 8 reports the number and percent of articles by year that reported income 
level and education level of families in the study. There do not appear to be any trends in 
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these data. Income level is reported between 1 9-26% of the time and education level 
between 20-30% of the time in the EI/ECSE literature included in this effort. 

Discussion and Recommendations for Future Efforts 
This article describes the literature review activities undertaken to contribute to 
the development of recommended practices in the field of early intervention/early 
childhood special education. The literature review consisted of articles from 48 peer- 
reviewed journals that reported research findings related to children with disabilities birth 
through 5 years, their families, those who work with them, or program policies. The data 
show which scientific method has been employed to study particular strands and specific 
practices. These data present the most comprehensive and systematic review of the 
EI/ECSE research literature to date about which topics and practices have been 
empirically studied and with which methodologies. 

Although this study is the most comprehensive of its kind to date, there are 
several limitations to the data. First, the criteria for selecting articles to be included in the 
review was limited to the 48 peer reviewed journals chosen by the management team and 
the strand chairs. That is, books, chapters, and conference proceedings were not included 
in this review. Second, studies selected for coding were based on the judgments of two 
investigators who read abstracts in the 48 journals over the 9 years. Third, due to resource 
limitations, the coding of all articles was completed by volunteers. However, inter-coder 
reliability of at least 85% was established with all coders prior to article coding and inter- 
coder agreement checks for recommended practices and strand placement were 
completed for 1/3 of the coded articles. 

There are several important implications of the study. First, the EI/ECSE 
knowledge base is founded on a wealth of research. In less than 1 0 years, over 1 000 peer 
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reviewed articles have been published with over 3/4 of them containing implications for 
practice. However, most journals in the field are not reporting significant numbers of 
empirical studies as defined in this project related to children with disabilities birth 
through five. Only about 25% of the journals reported 20 or more empirical studies 
related to this population over the 9 years reviewed. Finally, it is evident that the field of 
EI/ECSE has benefited from the rich array of research methods available to study 
particular questions. 

The analyses of the literature base reveals several possible directions for the field. 
First, future studies would benefit from clarifying study participants, their disability, the 
severity of the disability, ethnicity and any other characteristics that may impact on the 
usefulness and replicability of research findings. Journals might suggest standard 
classifications such as the IDEA definitions of disability, categories of severity of 
disability, and categories of study settings. Second, where it may be germane to 
replicability of results, clarity of family and service provider characteristics should be 
described. Finally, there are areas of practice that have less empirical research support 
than others, including interdisciplinary models, technology applications and personnel 
preparation. 

The authors offer several recommendations for future similar efforts to 
summarize the research base in EI/ECSE. Given the impetus to be evidence-based, these 
recommendations may lead to a more efficient method of review and an improved 
synthesis of the data-base. First, we recommend a review of the literature take place at 
least every 5 years. This might reduce the number of articles that need to be reviewed and 
coded. Second, such a review requires adequate resources. The efforts described in this 
paper are due to the many volunteer coders in the field. It is unlikely that this volunteer 
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resource can be tapped indefinitely! Third, coding forms should include more 
standardization of terms and few open-ended entry options to improve clarity and 
consistency and reduce interpretation errors. 

This is the first large-scale effort to summarize the literature base in EI/ECSE 
across all research methods. Such efforts are important to continue so that researchers can 
improve methods and reporting of settings, variables, and findings; scientific journals can 
endeavor to encourage more precision in studies and manuscripts; and funders and 
researchers can be evidence-based in identifying gaps or areas needing further attention 
in the on-going quest to identify which services and supports are most effective for which 
children, families, or the personnel who serve them. 
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Figure 1 . Data collection, analysis, and synthesis of recommendations from data sources 
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Table 1 

Number of Articles in Each Peer-Reviewed Journal Included in DEC Research Synthesis 



Journal Number of 

Articles 

American Journal of Occupational Therapy 32 

American Journal on Mental Retardation 50 

Archives of Physical Medicine and Rehabilitation 2 

Augmentative and Alternative Communication 1 5 

Behavior Modification 1 1 

Behavior Therapy 4 

Behavioral Disorders 1 

Child Development 6 

Child: Care, Health and Development 13 

Developmental Medicine and Child Neurology 40 

Developmental Psychology 5 

Diagnostique 6 

Early Childhood Research Quarterly 1 1 

Early Education and Development 21 

Education and Training in Mental Retardation and Developmental 1 8 

Disabilities 

Education and Treatment of Children 1 6 

Exceptional Children 47 
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Exceptionality 

Family Relations 1 3 

Infants and Y oung Children 1 2 

Infant-Toddler Intervention 45 

Journal of Abnormal Child Psychology 8 

Journal of Applied Behavior Analysis 55 

Journal of Applied Developmental Psychology 13 

Journal of Behavioral Education 1 1 

Journal of Communication Disorders 1 

Journal of Early Interventi on 1 02 

Journal of Emotional and Behavioral Disorders 7 

Journal of Marriage and Family 0 

Journal of Pediatric Nursing 1 3 

Journal of Pediatric Psychology 7 

Journal of Special Education 5 

Journal of Special Education Technology 0 

Journal of Speech, Language, and Hearing Research 26 

Journal of The Association for Persons with Severe Handicaps 32 

Journal of Visual Impairment & Blindness 18 

Language, Speech and Flearing Services in Schools 1 8 

Mental Retardation 22 

Merrill-Palmer Quarterly 0 

Neonatal Network 9 
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20 

Occupational Therapy Journal of Research 6 

Pediatric Physical Therapy 9 

Pediatrics 7 

Physical & Occupational Therapy in Pediatrics 12 

Physical Therapy 2 

Teacher Education and Special Education 1 

Topics in Early Childhood Special Education 73 

Volta Review 7 



Note: Journals in italics designate those that contributed 20 or more articles to the review. 
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Table 2 



Number and Percent of Articles Interpreted By Coders as Having at Least One 
Recommended Practice by Method 



Method 


N (%) 


Group Quantitative 


450 (54%) 


Single Subject 


184 (22%) 


Descriptive / Survey 


121 (15%) 


Qualitative 


74 (9%) 


Mixed Method * 


6 (1%) 


Total 


835 (100%) 



* Mixed method = some combination of group quantitative, qualitative, descriptive / 
survey and single subject methods 
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Number and Percent of Articles by Research Method Employed and Recommended Practices Strand Placement 



<N 

<N 
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Table 4 

Number of Articles by Method and Publication Year 



Method 


Year 




1990 


1991 


1992 


1993 


1994 


1995 


1996 


1997 


1998 


Group 


41 


43 


40 


60 


62 


51 


53 


67 


33 


Quantitative 


54.7% 


51.8% 


48.8% 


53.1% 


55.9% 


52.6% 


55.2% 


66.3% 


42.9% 




14 


13 


23 


33 


27 


21 


20 


13 


20 


Single Subject 


18.7% 


15.7% 


28.1% 


29.2% 


24.3% 


21.7% 


20.8% 


12.9% 


26.0% 


Descriptive / 


14 


22 


14 


16 


15 


13 


10 


6 


11 


Survey 


18.7% 


26.5% 


17.1% 


14.2% 


13.5% 


13.4% 


10.4% 


5.9% 


14.3% 




6 


5 


4 


4 


7 


9 


13 


14 


12 


Qualitative 


8.0% 


6.0% 


4.9% 


3.5% 


6.3% 


9.3% 


13.5% 


13.9% 


15.6% 


Mixed 


0 


0 


1 


0 


0 


3 


0 


1 


1 


Methods * 






1.2% 






3.1% 




1.0% 


1.3% 


Total 


75 


83 


82 


113 


111 


97 


96 


101 


77 



* Mixed method - some combination of group quantitative, qualitative, descriptive/survey and 
single subject methods 
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Table 5 

Number and Percent of Articles Reporting Disability Categories of Child Study Participants by 
Year 



Disability 










Year 










Category 






















1990 


1991 


1992 


1993 


1994 


1995 


1996 


1997 


1998 




6 


15 


10 


8 


15 


9 


8 


13 


15 


Autism 


5.1% 


12.8% 


7.7% 


4.9% 


8.9% 


6.1% 


5.6% 


7.3% 


15.3% 




14 


8 


13 


16 


13 


16 


12 


16 


6 


Cerebral Palsy 


11.9% 


6.8% 


10.0% 


9.9% 


7.7% 


10.9% 


8.4% 


9.0% 


6.1% 


Developmental 


15 


15 


20 


36 


23 


27 


24 


31 


22 


Delay 


12.7% 


12.8% 


15.4% 


22.2% 


13.7% 


18.4% 


16.8% 


17.4% 


22.4% 


Developmental 


5 


7 


9 


10 


9 


8 


10 


11 


8 


Disability 


4.2% 


6.0% 


6.9% 


6.2% 


5.4% 


5.4% 


7.0% 


6.2% 


8.2% 


Down 


9 


13 


14 


15 


21 


20 


16 


16 


7 


Syndrome 


7.6% 


11.1% 


10.8% 


9.3% 


12.5% 


13.6% 


1 1 .2% 


9.0% 


7.1% 


Emotional/ 


4 


5 


5 


6 


9 


2 


5 


10 


4 


Behavioral 


3.4% 


4.3% 


3.8% 


3.7% 


5.4% 


1.4% 


3.5% 


5.6% 


4.1% 


Disorder 




















Hearing 


5 


2 


6 


11 


8 


7 


10 


14 


5 


Impairment/ 


4.2% 


1.7% 


4.6% 


6.8% 


4.8% 


4.8% 


7.0% 


7.9% 


5.1% 


Deaf 




















Mental 


18 


15 


14 


15 


16 


13 


16 


17 


8 
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Retardation 


15.3% 


12.8% 


10.8% 


9.3% 


9.5% 


8.8% 


11.2% 


9.6% 


8.2% 


Speech/ 


9 


3 


10 


11 


16 


11 


12 


14 


4 


Language 


7.6% 


2.6% 


7.7% 


6.8% 


9.5% 


7.5% 


8.4% 


7.9% 


4.1% 


Impairment 




















Visual 


5 


9 


7 


6 


6 


5 


5 


10 


4 


Impairment/ 


4.2% 


7.7% 


5.4% 


3.7% 


3.6% 


3.4% 


3.5% 


5.6% 


4.1% 


Blindness 






















28 


24 


22 


27 


32 


29 


25 


26 


15 


Other 


23.7% 


20.5% 


16.9% 


16.7% 


19.0% 


19.7% 


17.5% 


14.6% 


15.3% 
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Table 6 

Number and Percent of Articles Reporting Severity of Disability by Year 



Year 





1990 


1991 


1992 


1993 


1994 


1995 


1996 


1997 


1998 


Articles 


55 


51 


48 


71 


66 


51 


61 


54 


42 


Reporting 


73% 


61% 


59% 


63% 


59% 


53% 


64% 


53% 


55% 


Severity 
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Table 7 



Number and Percent of Articles Reporting Study Setting Data 





Number / Percent of 


Study Setting 


Articles 




225 


Home 


23.4% 




230 


Hospital/Clinic 


23.9% 




170 


Inclusive/Integrated Classroom 


17.7% 




130 


Separate or Segregated Classroom 


13.5% 




33 


Experimental Settings/Separate from peers 


3.4% 




152 


Other - Survey/Interview 


15.8% 




23 


Other - Not clear 


2.4% 


Totals 


963 
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Table 7 



Number and Percent of Articles Reporting Study Setting Data 





Number / Percent of 


Study Setting 


Articles 




225 


Home 


23.4% 




230 


Hospital/Clinic 


23.9% 




170 


Inclusive/Integrated Classroom 


17.7% 




130 


Separate or Segregated Classroom 


13.5% 




33 


Experimental Settings/Separate from peers 


3.4% 




152 


Other - Survey/Interview 


15.8% 




23 


Other -Not clear 


2.4% 


Totals 


963 
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Figure 7 (Alternate) 
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Table 8 

Number and Percent of Articles Reporting Family Income and Education Levels by Year 





Year 




1990 


1991 


1992 


1993 


1994 


1995 


1996 


1997 


1998 


Articles 




















Reporting 


16 


16 


19 


29 


22 


19 


23 


22 


18 


Family Income 


21% 


19% 


23% 


26% 


20% 


20% 


24% 


22% 


23% 


Level 




















Articles 




















Reporting 


20 


18 


22 


29 


22 


25 


34 


34 


26 


Family 


27% 


22% 


27% 


26% 


20% 


26% 


35% 


33% 


34% 


Education 




















Level 
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Evidence-based Practice in Early Intervention/Early Childhood Special 
Education: Single Subject Design Research 
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and 
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Abstract 

The purpose of this study was to examine the strength of scientific 
evidence from single subject research that underlies the DEC 
Recommended Practices. From the 1991 to 1998 research literature, 184 
studies met the inclusionary criteria for this review. In general, the 
literature provided positive, and in some cases very strong, scientific 
evidence for the effectiveness of practices from the Child-Focused Strand 
of the DEC Recommended Practices. Researchers used multiple baseline, 
multiple probe, alternating treatment, and a combination of designs most 
frequently. Treatment effects were replicated an average of 5.5 times per 
study, and treatments appeared to be implemented with a high degree 
of fidelity. Researchers less often provided evidence for maintenance, 
generalization, and social validity of treatments. Recommendations for 
future research are offered. 
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Evidence-based Practice in Early Intervention/Early Childhood 
Special Education: Single Subject Design Research 
In the field of general education and in special education, great 
emphasis has been placed on basing educational practice on scientific 
evidence (Carnine, 1995; Shavelson & Towne, 2002). In the sub-discipline 
of early intervention/early childhood special education (EI/ECSE), such an 
approach is also critically important (Guralnick, 1999). Yet, the synthesis of 
scientific evidence that supports educational and early intervention 
practices may be limited by a somewhat constrained definition of 
"science”. For many years, research employing single subject 
experimental designs has made substantial contributions to the scientific 
evidence underlying effective EI/ECSE practice (Patrick, Mozzoni, & 

Patrick, 2000; Strain et al„ 1992). However, in attempts to synthesize the 
scientific literature on EI/ECSE, single subject research methodology has 
often been omitted (Casto & Mastropieri, 1986; Strain & Smith, 1986). 

Perhaps because the roots of single subject research methodology 
are in applied behavior analysis (Baer, Wolf, & Risley, 1968), clinical 
psychology (Kazdin, 1975), and special education (Haring & Lovitt (1967), 
single subject research rarely appears in the general education literature. 
In fact, in a report of a recent committee convened by the National 
Academy of Sciences (NAS) to review the status of scientific research in 
education and make recommendations for future research, single subject 
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methodology was never mentioned (Shavelson & Towne, 2002). However, 
rigorous single subject methodology (Kazdin, 1982; Tawney & Gast, 1983; 
Wolery & Dunlap, 2001), certainly meets the principles of scientific 
research established by the NAS committee, which were; a) conducting 
an empirical investigation, b) linking findings to a theory of practice, c) 
using methods that permit direct investigation, d) providing a coherent 
chain of reasoning, and e) replicating and generalizing across studies. 

In the related discipline of clinical psychology, single subject design 
research does appear in the literature. Division 12 of the American 
Psychological Association established criteria for determining the 
empirical evidence that supported practice in psychotherapy and clinical 
psychology (Lonigan, Elbert, & Johnson, 1998) and included criteria for 
single subject research. Practices could be judged "well-established" if a 
large series of single-case designs (n>9) used good experimental design, 
compared the intervention to another treatment or condition, had a 
treatment manual, and described participants clearly. A practice could 
be classified as "Probably Efficacious” if a smaller set of studies (n>3) met 
the criteria just described. 

The issue of rigorous experimental methodology is important in 
determining the degree to which practices have a scientific basis. In 
single subject methodology, a number of features represent a rigorous 
application of the methodology. These include a) building evidence for 
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effectiveness through within subject or between subject replications of a 
treatment effect that demonstrates improvement in children behavior in 
the desired direction (Wolery & Dunlap, 2001 ); b) assessing the reliable 
application of the treatment, which is referred to as fidelity, integrity or 
implementation of treatment (Gresham, Gansel, & Kurtz, 1993; Peterson, 
Homer, & Wonderlich, 1982; LeLaurin & Wolery & 1992); c) systematic and 
quantitative assessment of the dependent variable (Kazdin, 1977); d) 
maintenance of treatment effects across time (Kazdin, 1982); and d) 
generalization of treatment effects across settings, persons, and/or 
behaviors (Stokes & Baer, 1977). In addition, high quality, rigorous single 
subject methodology includes judgments of the social validity (i.e., the 
social importance) of the treatment (Wolf, 1978; Schwartz & Baer, 1991 ). 

Given that single subject research informs the practice of EI/ECSE 
(Strain et al., 1992; Patrick et al., 2000), then any attempt to summarize the 
scientific evidence that underlies practice in EI/ECSE must include single 
subject design research. Like all paradigms of scientific inquiry, single 
subject design studies vary in their rigor and sophistication, so simply 
documenting the number of studies that support a specific practice 
provides little information about the quality or rigorous nature of the 
scientific support. In the current study, the aim was to provide a strength- 
of-evidence analysis in which number of replications and study rigor could 
be summarized to yield conclusions about a) the degree of support for 
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individual EI/ECSE practices and b) the quality of single subject design 
research in EI/ECSE. Such an analysis would also provide information 
about gaps that may exist in the empirical support for practice and/or the 
quality of the research methodology employed, which in turn could 
inform a future research agenda. 

Methods 

To summarize the research on recommended practices in EI/ECSE, 
a task force of the Division for Early Childhood accessed research 
published in 48 peer-reviewed journals between the years of 1991 and 
1998. This process is described in detail in Smith et al. (in press). To be 
included in the current review, articles had to contain original research 
and involve children with disabilities from birth to 5 and/or their families, 
personnel working with children with disabilities and/or their families, or 
policies or systems change procedures. Studies had to include 
quantitative or qualitative data that demonstrated the effectiveness of a 
practice or set of practices, and were excluded if they primarily were 
psychometric evaluations of assessments, descriptive case studies 
involving only a single individual or program, or descriptions of children’s 
development or family functioning in the absence of a specific 
intervention or practice being implemented. Studies were categorized as 
group design, single subject design, or qualitative design. In this paper, 
only information on single subject design will be presented. The criteria for 
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inclusion in the single subject design data base were that the study had to 
have at least one demonstration of experimental control of the 
independent variable and that it employed a standard single subject 
design methodology (Tawney & Gast, 1983) . Thus, A-B designs that 
involved only a baseline and implementation of the treatment without a 
withdrawal of treatment phase, replication across settings, behaviors, or 
participants were excluded. In the initial identification of studies, 208 
articles were identified. Of this number, 184 meet the criteria for inclusion 
in the single subject design database. 

Coding Procedures 

All single subject research studies were coded using Forms A and B. 
Form A provided a generic description of the articles [see Smith et al. 
(2002) for a description of Form A] and was used uniformly across all 
articles in the Recommended Practices database. Form B-Single Subject 
Specialized Coding Sheet was used for all single subject design articles. 
This form was developed by the co-authors of this study, pilot tested with a 
sample of single subject design studies, and reviewed by researchers 
knowledgeable about single subject design methodology. In the Form B 
evaluation, coders rated descriptive parameters of the single subject 
methodology employed and the results. These parameters included the 
type of single subject design, whether the study took place in a 
group/integrated setting or in an individual/therapeutic and separate 
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setting, whether the agent of intervention was an adult or peer, how 
many replications of the intervention effect occurred, and a narrative 
description of the dependent variable(s) and independent variable(s). 
Eight additional items were designed to produce an index of outcome 
believability by rating each study on design features that indicated high 
quality, rigorous single subject studies. The eight quality design features 
and their associated rating schemes were as follows: 

A) Independent variable assessment - Yes =1 ; No = 0 

B) If yes, fidelity of intervention was - High = 2; Medium = 1 ; Low = 0 

C) Evidence of improvement over time - Yes = 1 ; No = 0 

D) Evidence of intervention maintenance across time - Yes = 1 ; No = 0 

E) Evidence of intervention generalization - Yes = 1 ; No = 0 

F) Social validity assessments included - Yes = 1 ; No = 0 

G) Outcome acceptability (consumer satisfaction included) - Yes = 1; 
No = 0 

In addition, after these ratings were completed, the coders summarized 
the practice(s) that was/were supported by the data presented in the 
article. This information was then used to inform the development of the 
DEC Recommended Practice Indicators (Smith, McLean, Sandall, Snyder, 
& Broudy, 2000). 

Coders of Single Subject Design Articles 
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The two co-authors of this study recruited 16 individuals with direct 
experience conducting and reviewing single subject research to serve as 
coders for this study. Prior to reviewing any articles for this analysis, coders 
read the rating forms and practiced coding a standard article. In 
individual telephone calls, the co-authors discussed the rating items and 
compared the coder’s rating with the standard rating established by the 
co-authors. Discrepancies were discussed and resolved. Coders then 
rated a second article, which was sent to the co-authors. To be included 
as a coder, potential raters had to meet an 85% inter-rater agreement 
criterion (between the coder and one co-author). After meeting this 
criterion, coders were assigned 10-20 articles. Inter-rater agreement was 
then conducted on 33% of the single subject design articles. Inter-rater 
agreement exceeded 85% across item ratings 

Results 

The initial review of the literature revealed a total of 215 single 
subject design articles published during the 1990s that provided support 
for recommended practices in early intervention/early childhood special 
education (See Table 1 ). The total number of studies exceeded the 
number of articles noted previously because some articles contained 
more than one study. All of the studies were associated with DEC 
Recommended Practices in the Child-Focused Intervention Strand. The 
average number of studies associated with a specific recommended 
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practice was 9.97, with a range from one study supporting C26-lntensity of 
Instruction to 32 studies supporting C24-Prompting and Prompt Fading. 

The greatest number of single subject design studies supporting practices 
were for C24-Prompting and Prompt Fading, C27-Functional Assessment 
and PBS, C21 -Consequences Support Play, Engagement, Learning, C22- 
Naturalistic Teaching Approaches, C23-Peer-mediated Strategies, and 
Cl 2-Practices Individualized. 

Designs 

Researchers employed all the major single subject designs in their 
studies of early intervention/early childhood practices. The frequency of 
each type of design appears in Figure 1 . The design that researchers 
employed most frequently was the multiple baseline design. The multiple 
probe design, a variation on the multiple baseline design, and the 
alternating treatment design, most often used to compare treatments, 
occurred relatively frequently. The two designs that required a withdrawal 
of treatment or conditions which led to a reversal of treatment effects 
were used less often. Both types of designs, however, still appeared in 
over 20 studies. Also, researchers used variations on the standard designs 
(i.e., the Other category) frequently to address specific questions. Last, 
the simultaneous treatment design, a design having very specific 
experimental requirements, was only used once in this data set. 
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Independent Variable 

Two important features of single subject designs are related to 
treatment fidelity. Fidelity is assessed by measuring the degree to which 
the independent variable is implemented as planned. In the single 
subject design studies from this database, researchers assessed the 
independent variable in 50% of the studies. When the independent 
variable was assessed, the fidelity of treatment (i.e., implementation) was 
rated as high (2), medium (1 ), or low (0). The average ratings for studies 
grouped by recommended practices appear in Table 1 . The average 
fidelity of treatment rating for the entire sample was 1 .79, indicating an 
overall high implementation of intervention, when this variable was 
assessed. 

Experimental Control and Replications 

The feature of single subject designs that makes them experimental 
is the demonstration of a functional relationship between the 
independent and dependent variable (Baer, Wolf, & Risley, 1968). This 
functional relationship demonstrates experimental control through within- 
subject and between-subject replications of treatment contrasts (i.e., a 
change in the dependent variable occurs when treatment is 
implemented after a baseline condition). Two types of data address this 
issue of experimental control. Raters identified the number of replications 
of treatment conditions (i.e., implementation of treatment after a 
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baseline) that occurred in each study. Across studies, the mean number 
of replications was 5.5, with a range from 2.25 to 14.5 replications across 
individual practices. Improvement in children’s performances within 
phases occurred in nearly 78% of the studies. The means ranged from 50% 
for the two studies associated with C13-Meaningful Outcomes to 100% for 
the two studies associated with Cl 5-Functional Outcomes, the two studies 
associated with C20-Normalized Interventions, and the two studies 
associated with C26-lntensity of Instruction. Together, these data 
indicated that experimental control was tested often through multiple 
replications and yielded a relatively high degree of success. 

Maintenance and Generalization 

The assessment of the durability, continuity, and transfer of 
treatment effects were assessed through maintenance of treatment 
effects (i.e., changes in behavior continue after intervention has ended) 
and generalization (i.e., behavior changes occur in other settings, with 
other people, or are reflected in changes in other types of behavior). In 
this study, raters determined if there was evidence for maintenance or 
generalization. Unfortunately, we did not differentiate between studies 
that assessed maintenance and generalization and did not find it and 
studies that did not assess maintenance and generalization. So, these 
data are somewhat limited and perhaps conservative in their estimates. 
Raters found that 39% of the studies provided evidence of maintenance. 
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This figure ranged from 0% for C2-Social Ecology Structured, Cl 1 -Foster 
Social Relationships, and C26-lntensity of Instruction to 67% for C25- 
Embedded Instruction and 60% for Cl 2-lndividualized Interventions and 
Cl 5-Functional Outcomes. 

The average percentage of studies providing evidence of 
generalization was 43.89%, again with a large range. A mean of 0% 
occurred for C14-Data-Based Decision Making, C20-Normalized 
Interventions, and C26-lntensity of Instruction. In contrast, 80% of the 
studies associated with Cl 5-Functional Outcomes, 68% of the studies 
associated with C22-Naturalistic Interventions, and 67% of the studies 
associated with C25-Embedded Intervention provided evidence for 
generalization. In general, authors tended to provide more evidence for 
generalization than for maintenance. 

Social Validity 

Social validity is a judgment of the social importance of intervention 
goals, procedures, and effects (Wolf, 1978). In this analysis, raters assessed 
whether researchers measured the social validity of their treatment 
procedures (i.e., acceptability) or outcomes. The means for both types of 
social validity were quite low. The average across groups of studies was 
1 5.39% for treatment acceptability. Eight recommended practices had 
no studies reporting treatment acceptability. For two Recommended 
Practices with small numbers of studies (Cl 1 -Foster Social Relationships 
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and C26-lntensity of Instruction), all (100%) studies reported treatment 
acceptability. 

The judgment of the social importance of the effects of 
interventions is a second form of social validity. The mean percentage 
(27.19%) was slight higher for this measure than for treatment 
acceptability. Three groups of recommended practices also reported no 
measures of outcome social validity, C8-Variety of Intervention, C9- 
Natural Environment, and C20-Normalized Interventions. Like the 
Treatment Acceptability measurement, all of the studies associated with 
Cl 1 -Foster Social Relationships and C26-lntensity of Instruction reported 
Outcome Social Validity, as did 67% of the studies associated with C2- 
Social Ecology. 

Discussion 

The purpose of this study was to examine the scientific evidence for 
the effectiveness of practices in EI/ECSE research provided by single 
subject design research. A second purpose was to examine the quality of 
the research methodology of the studies that supported EI/ECSE 
practices. As noted previously, Division 12 of APA established single 
subject research criteria for evidence-based practice, which could be 
used to determine empirical support for EI/ECSE practices. Six of the DEC 
Recommended Practices in the Child Focused Strand met the number of 
studies criterion (>9) for “well established” efficacious practice (Cl 2- 
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Individualized Intervention, C21 -Consequences Support 
Play/Engagement/Learning, C22-Naturalistic Teaching Approaches, C23- 
Peer-mediated strategies, C24-Prompting and Fading, C27-Functional 
Assessment and Positive Behavior Support). From this body of research, 
teachers may have confidence that when these practices are 
appropriately and consistently implemented with children, they are highly 
likely to produce positive effects in the settings in which they are applied. 

A second set of practices met the APA Division 12 frequency of 
studies criterion (>3) for being "probably efficacious". These practices 
included: planning and organizing the Cl -Physical Ecology and C2-Social 
Ecology of the classroom to promote learning, C3- Planning Routines and 
Transitions Within the Class, C8-Teaching Skills in Different Activities, C9- 
Teaching Skills in Natural Environments, Cl 4-Using Data to Make Decisions, 
C15-Selecting Functional Outcomes, Cl 6-Responding Contingently to 
Children’s Behavior, Cl 7-Using Different Teaching Approaches for 
Different Phases of Learning, C 18-Involving Multiple Individuals and 
Materials in Teaching, Cl 9-Planning Instruction in which Children Apply 
the Skills, and C25-Embedding Instruction in Ongoing Activities in the 
Classroom. From the single subject research literature, teachers can have 
some confidence that these instructional practices will yield positive 
effects when implemented consistently, although the practices are not as 
strongly supported by the literature as are those mentioned previously. 
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A third set of practices received limited support from the single 
subject design literature (i.e. ( number of studies <3). These included C 1 1 - 
Fostering Social Relationships, C13-Meaningful Outcomes, C20-Using 
Normalized Intervention Approaches, and C26-lntensity of Intervention. In 
addition, several strategies were not supported by the single subject 
design literature: C4-Structuring Play Routines, C5-Promoting Cultural and 
Linguistic Awareness, C6-Meeting Quality Standards, C7-Providing Safe 
Environments, and Cl O-Promoting Child-Initiated Engagement. These 
practices are valued within the field and probably draw their support from 
group design and qualitative design research studies. Given that one aim 
of this study was to identify gaps in the single subject design literature, 
certainly a direction for future research could be to employ single subject 
methodology to further substantiate the effectiveness of these practices. 

Several features of the data presented in this study indicate strong 
general support for the DEC Child Focused Recommended Practices. On 
the average, the treatment effect was replicated 5.5 times within a study. 
The experimental nature of single subject research is based on multiple 
replications of the treatment effects, which were occurring in this set of 
studies. Although on the average only half of the studies assessed fidelity 
of treatment, when it was assessed the treatments appeared to have 
been implemented with a high degree of fidelity (i.e., a mean rating of 
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1 .64 out of 2.0). Plus, in the substantial majority of studies, improvement in 
students’ performance occurred across phases. 

Evidence for maintenance and generalization of treatment effects 
were less compelling. On the average, only 39% of the studies provided 
evidence for the maintenance of treatment effects after the intervention 
ended. Although the range extended up to 67% for some practices, in 
the majority of studies researchers omitted the important measurement or 
documentation of maintenance. Likewise, only about 44% of the studies 
provided evidence of generalization of treatment effects. Although this 
percentage ranged higher for some practices (e.g., 68% for C22- 
Naturalistic Teaching Strategies) and the average percentage was higher 
than for maintenance, the majority of studies still did not provide 
evidence for generalization. Unfortunately, because of the coding system 
used, these data do not discern between studies in which no 
maintenance or generalization data were collected and studies in which 
maintenance and generalization data were collected but treatment 
effects did not maintain or generalize. A clear recommendation for the 
future use of single subject research in EI/ECSE is that investigators 
incorporate into their study procedures plans to assess maintenance and 
generalization as well as instructional procedures to support it. 

Social validity is another dimension of single subject methodology 
that reflects the rigorous nature of the experimental design. In this study. 
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coders documented the assessment and evidence for two forms of social 
validity: acceptability of treatment procedures and social importance of 
outcomes. Overall, assessment of both types of social validity occurred in 
only a moderate percentage of the studies (i.e., 15% and 27% of the 
studies respectively). Again, a clear recommendation emerging from 
these data is that researchers employing single case design in EI/ECSE 
need to be more vigilant in their assessment of the social validity of 
treatment procedures and outcomes (Schwartz & Baer, 1991). 

The analysis of the types of single subject experimental designs 
indicated that researchers selected multiple baseline designs much more 
frequently than any other design, with multiple probe designs also being 
used often. The advantage of multiple baseline designs is that the 
researcher does not have to withdraw treatment or reverse positive 
performances for children in order to demonstrate an experimental effect. 
The ethics of withdrawing treatment, once a practice has produced 
positive results, is troublesome and may prevent researchers from 
employing this design. However, it is important to note that multiple 
baseline/probe designs are not without ethical issues. In some designs, 
children in the second and third tiers of the multiple baseline may remain 
in a baseline phase for an extended time. Another type of design 
employed relatively often was the alternating treatment design, which 
allows researcher to compare the relative strengths of treatments. This 
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design also does not require a withdrawal of effective treatment, but 
rather an ongoing comparison of two treatments. We anticipate that both 
the multiple baseline, multiple probe, and alternating treatment designs 
will continue to be designs of choice for EI/ECSE researchers in the future. 

In conclusion, the single subject design research of the 1 990s 
provides a scientific basis for some of the standard practices in EI/ECSE. 
The studies reviewed were related specifically to child focused practice. 
Research employing other methodologies has been used to support 
practices in other strands as well as provide complementary support for 
practices in the Child-Focused strand. As a body of research, the quality 
of the basic design and methodology appeared strong, although 
evidence for maintenance, generalization, and social validity was lacking 
and should be a focus for future research. 
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Figure Caption. Frequency of different types of single subject designs 
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Abstract 

Findings are reported related to the research methods and statistical techniques used in the 450 
group quantitative studies examined as part of the literature review portion of the Division for 
Early Childhood Recommended Practices project. Twelve trained coders used an investigator- 
developed coding form to analyze studies across seven major dimensions: (a) sampling 
procedures, (b) variable selection, (c) variable definition, (d) measurement integrity, (e) 
treatment fidelity, (f) statistical analyses, and (g) magnitude-of-effect reporting. Results 
suggested that the methodological rigor of the quantitative research used to inform recommended 
practices was not uniformly convincing and compelling. Implications are offered related to the 
strength of empirical support for recommended practices and the conduct and reporting of future 



research. 
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Examination of Quantitative Methods Used in 
Early Intervention Research: Linkages with Recommended Practices 
Efforts to produce recommended practices involve integration of the “best available” 
research evidence with knowledge gained through experience. This integration should occur in a 
context that addresses the social validity of the proposed practices. Peters and Heron (1993) 
offered a set of five questions that should guide efforts to produce recommended or best 
practices: (1) Does the practice have a sound theoretical base? (2) Is the methodological integrity 
of the research convincing and compelling? (3) Is there consensus within the existing literature? 
(4) Is there evidence that desired outcomes are consistently produced? (5) Is there evidence of 
social validity? Each of these five questions was considered in the formulation of the Division 
for Early Childhood (DEC) Recommended Practices and the procedures by which these 
questions were operationalized is described by Smith, McLean, Sandall, Snyder, and Broudy 
(2000) and Smith et al. (2002). 

The methodological integrity shown in early intervention (El) research has been 
complimented and criticized simultaneously over its history. Compliments generally have 
accrued when the complexity of conducting applied research on complicated issues with low- 
incidence or heterogeneous populations is considered and the incremental progress related to 
addressing early intervention efficacy is demonstrated (Guralnick, 1997; Simeonsson, 1985; 
Strain & Smith, 1986). Criticisms typically have revolved around issues associated with lack of 
methodological rigor, including poor design, imprecise and insensitive measurement, and 
inadequate statistical analyses (e.g., Casto & Mastropieri, 1986; Dunst & Rheingrover, 1981; 
Farran, 1990, 2000). 
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To date, compliments and criticisms about early intervention research primarily have 
been associated with published reviews related to whether early intervention is effective for 
young children with disabilities or those at-risk for disabilities. Several influential researchers 
have suggested that El research accumulated over the past 30 years permits moving beyond 
questions related to whether services and supports should be provided to questions related to 
which services and supports are most effective for which children and families (e.g., Guralnick, 
1997). As the field develops a clearer vision of the types of research that should be conducted 
(see Wolery & Bailey, 2002) and continues to use best available evidence to formulate 
recommendations for practice, a review of past methodological approaches and trends may offer 
constructive implications for the future (cf. Shulman & Margalit, 1985). Further, specific 
guidelines for reporting research findings have been promulgated recently in the Journal of Early 
Intervention (McWilliam, 2000; Snyder, 2000; Wolery & Dunlap, 2001) and elsewhere (e.g., 
APA, 2001; Wilkinson & the Task Force on Statistical Inference, 1999). The extent to which 
research reported in El-related journals corresponds with these guidelines should be informative 
for authors, editors, and consumers of research. 

The purpose of this article is to present findings related to the research methods and 
statistical techniques used in the 450 group quantitative studies examined as part of the literature 
review portion of the DEC Recommended Practices project (see Smith et al., 2002 for additional 
details about the overall literature review and associated coding procedures). The objectives of 
this quantitative review were to evaluate the scientific rigor of each study and to help determine 
recommended practice(s) supported by group quantitative research evidence. To evaluate rigor, 
we examined sampling procedures, variable selection, variable definitions, measurement 
integrity, treatment integrity, and statistical analyses employed. 
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Method 

Articles Included in the Review 

As described by Smith et al. (2002), empirical research published in 48 peer-reviewed, 
professional journals relevant to the field of El from 1990 through 1998 were included in the 
overall review. Inclusion criteria were that each article (a) be an original research report, and (b) 
either involve children with disabilities birth through 5, families of children with disabilities birth 
through age 5, personnel who serve these children and families, or policies or systems change 
procedures that support effective practice with these groups. Studies not eligible for inclusion 
were those that reported the development and testing of measurement instruments used in early 
intervention, normative reports, “« of 1” descriptive case studies, and developmental research. 
Articles included in the group quantitative category for review and subsequent coding had to 
meet the inclusion criteria described above and, in addition, had to be classified as using one of 
three experimental designs (pre-experimental, quasi-experimental, true experimental) or as 
causal-comparative research (Campbell & Stanley, 1966; Gay & Airasian, 2000). 

Coders of Quantitative Articles 

Members of the DEC Recommended Practices project management team recruited 
coders. Forty-two individuals who were members of the Division for Early Childhood (DEC) 
volunteered to be trained and to code articles. Of these 42 individuals, 12 met criteria for serving 
as quantitative coders because they had established expertise in experimental group design and 
met or exceeded inter-coder agreement standards established by the project investigators. The 
majority (n - 8 ) of quantitative coders were members of the DEC Research Committee. Three of 
the coders were male; nine were female. Eleven coders were employed as faculty in various 
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institutions of higher education throughout the United States and one coder was a doctoral 
student in early childhood special education. 

Coding Forms 

Two quantitative methodology consultants from the project management team developed 
a specialized coding form (Form B-Quantitative) designed to permit recording of variables 
associated with scientific rigor in quantitative studies. Form B-Quantitative supplemented the 
generic coding form (Form A), which was used uniformly across all studies included in the 
literature review (see Smith et al., 2002 for descriptions of Form A). Form B-Quantitative 
prompted recording of information about types of sampling procedures used, major classes of 
variables studied, measurement tools used, types of psychometric integrity data reported for 
study measures, treatment integrity information, types of inferential statistical analyses used to 
address major study hypotheses, and statistics related to result importance (i.e., magnitude-of- 
effect indices). 

The initial version of Form B-Quantitative contained 55 coding categories. A quantitative 
methodology consultant and a project investigator piloted the initial form by separately coding 
five articles which met the inclusion criteria for quantitative group design studies but which were 
not included in the literature review because they were published before 1990. Form A and Form 
B-Quantitative were completed for each study. Responses to each of the coding categories were 
compared across the two coders and inter-coder agreement was evaluated based on the number 
of agreements/number of agreements plus disagreements. Agreement averaged 87% across the 
five studies coded (range 81% to 92%). Disagreements were discussed and revisions to Form A 
and Form B-Quantitative coding forms were made. The final version of Form B-Quantitative 
contained 53 coding categories. (A copy of this coding form is available from the first author). 
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Procedures 

A project investigator, project coordinator, and graduate student worker identified articles 
that met inclusion criteria for quantitative group studies. A quantitative methodology consultant 
subsequently verified the appropriateness of these articles for inclusion in the quantitative review 
and trained the coders. Training of coders involved review of a quantitative coding training guide 
developed by the quantitative methodology consultants that (a) described coding categories, (b) 
provided opportunities to practice coding using short excerpts from published literature, and (c) 
required coding of an entire article using Form A and Form B-Quantitative. After training, 
coders were required to code an additional quantitative group design article, which met inclusion 
criteria but was published before 1990, to establish inter-coder agreement of 85% or greater with 
an expert standard (i.e., quantitative methodology consultant). 

Most coders (n = 8 of 12) achieved inter-coder agreement of 85% or greater after coding 
two articles (range 86% to 99%). Those coders who did not achieve inter-coder agreement 
standards after coding two articles were provided additional articles to code until they reached 
inter-coder agreement of 85% or greater. Inter-coder agreement of greater than 85% was required 
before a coder was assigned articles to review from the pool of eligible studies. 

Following training, coders were assigned between 10 and 40 articles to code using Form 
A and Form B-Quantitative. Coders were sent copies of each article assigned to them, sufficient 
copies of Form A and Form B, and information about how to contact an investigator or 
methodology consultant if questions arose while coding. 

Coders were asked to complete coding of their assigned articles within 1 to 3 months. 
Two coders subsequently were unable to complete coding of all articles assigned to them within 
established timelines. These articles (n = 78) were re-assigned for coding to a quantitative 
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methodology consultant and to a project investigator who participated in the coder training and 
reached established inter-coder agreement standards. 

Data Analyses 

Data from the coding forms were entered into a spreadsheet program (Microsoft Excel™) 
at the primary research site and later transferred to a cooperating research site for integration into 
Microsoft Access™ databases. Databases were reviewed for potential errors (e.g., duplicate 
records, impossible values) and data entry errors were resolved by consulting the coding form, 
or, if necessary, by review of the research article associated with the coding form. The 
quantitative database contained 450 records (articles). These records were queried to produce the 
results described below. 

Results 

Sampling Procedures 

In 47 (10.4%) studies, participants were randomly selected from defined populations. 
Participants in the remaining 403 studies were samples of convenience. Random assignment to 
groups occurred in 9.7% (n = 44) of the studies with six of these studies involving a control 
group (i.e., group that received no intervention). Four studies (0.8%) used both random selection 
and random assignment procedures. Thirty-eight (8.4%) studies reported use of a control group 
with or without random assignment to a control condition. 

Matching participants on various attribute variables occurred in 106 (23.5%) of the 
studies. Covariate controls to adjust for non-equivalence of groups were used in 51 (1 1.3%) of 
the studies. 
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Variable Selection 

At least one independent or attribute variable was identified in 415 studies. The 
remaining 35 studies involved one-group, pre-experimental designs in which all participants 
received the same treatment or were exposed to the same condition. The majority of independent 
or attribute variables (n = 255, 61.4%) selected for study were related to children. Table 1 shows 
the numbers and representative examples of independent or attribute variables selected for study 
organized under the seven recommended practices strand categories: child, family, 
policy/systems, personnel, technology, assessment, and interdisciplinary models. 

Coders were asked to list dependent variables for each study and identify associated 
dependent measures. One dependent variable (DV) was identified in 182 articles, two DVs in 
102 articles, three DVs in 84 articles, and four or more DVs in 82 articles. 

Variable Definitions 

Primary dependent measures reported for the 450 articles were recorded and 
subsequently categorized into one of four groups: (a) investigator developed, (b) commercially- 
available, (c) published in research or limited circulation, and (d) measuring devices. 
Investigator-developed measures were the most frequent (n = 242) type of measure used as a 
primary dependent measure. Table 2 shows the frequency and representative examples of 
dependent measures in each category. 

Measurement Integrity 

In 255 (56.6%) studies, information was provided about at least one type of reliability of 
scores for primary dependent measures. In 37.5% (n = 169) of the studies, score reliability 
estimates were based on data obtained from study samples. At least one validity estimate was 
reported in 92 (20.4%) of the studies. In 17 (3.7%) studies, score validity estimates were based 
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on data obtained from study samples. Table 3 shows the frequencies associated with the different 
types of reliability and validity estimates reported for primary dependent measures and the 
number of these estimates that were based on study sample data. 

Measures of inter-observer agreement or inter-observer reliability for primary dependent 
measures were gathered in 158 (81%) of 195 studies involving observational data collection. The 
most common agreement/reliability indices used were percent agreement (94 studies), kappa (47 
studies), and Pearson product-moment correlation coefficient (12 studies). Intraclass correlation 
or generalizability coefficients were reported in 1 2 studies. 

Treatment Integrity 

Data were provided about the schedule of implementation (e.g., frequency, intensity, 
duration) for the independent variables or treatment provided in 217 (48.2%) of the studies 
reviewed. Coders judged that 57 (12.67%) of the 450 studies provided explicit information 
related to treatment integrity. Examples included training with feedback for those who were 
responsible for implementation of the independent variable, procedural checklists, and scripts. 
Statistical Analyses 

Statistical analyses employed in the 450 studies were coded under five broad categories: 
(a) univariate parametric (e.g., t-tests, ANOVA, multiple regression, Pearson product-moment 
correlation), (b) univariate nonparametric (e.g., chi-square, Spearman rank correlation, Mann 
Whitney U), (c) multivariate parametric (e.g., MANOVA, discriminant analysis, canonical 
correlation analysis), (d) multivariate nonparametric (e.g., logistic regression), and (e) other (e.g., 
structural equation modeling, hierarchical linear modeling). Table 4 shows the five categories of 
statistical analyses and the numbers of studies using these techniques per year. Univariate 
parametric techniques (e.g., t-tests, ANOVA, Pearson product-moment correlation, multiple 
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regression analysis) were the most common class of statistical analyses used across all years 
included in the review (i.e., 1990 through 1998). 

Eighty studies (17.7%) reported magnitude-of-effect indices in association with the 
primary study hypothesis. Table 5 lists the number and type of magnitude-of-effect indices 
reported under four major categories: (a) variance-accounted-for, uncorrected; (b) variance- 
accounted-for, corrected; (c) mean difference; and (d) other. 

Discussion 

The findings reported in this paper offer a descriptive analysis of the scientific rigor of 
450 group quantitative El research articles published in 48 journals from 1990 through 1998. 
When linked with efforts to produce recommended practices for the field, the results of the 
analysis help address questions about whether the methodological integrity of the group 
quantitative research is convincing and compelling. The findings reported in this paper also offer 
implications for the conduct of future research and the reporting of research findings. 

Design and Sampling Procedures 

The majority of research analyzed employed causal-comparative or quasi-experimental 
designs. These types of designs produce evidence at an individual study level that is less 
convincing and compelling than true experimental designs, particularly in relation to establishing 
cause-effect relationships and directly informing a recommended practice. As Kaufman (1990) 
noted, however, the contribution of research to practice is rarely achieved from a single study, 
even a true experimental design study. To inform the development of recommended practices, 
evidence must accumulate across studies and findings from a single study must be synthesized 
and integrated with knowledge generated from other studies addressing similar questions. 
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The lack of rigor found in the quantitative designs used across the reviewed articles may 
not reflect poor science or scientists as much as the reality of conducting field research on 
complicated questions, often with intact groups (Strain & Smith, 1986). Research in early 
intervention continues to be primarily field based, where random assignment and establishment 
of control groups often are difficult to achieve (Heckenlaible-Gotto & Choi, 1993). 

Realities associated with conducting field-based research should not be used to excuse 
the use of quantitative group designs of poor quality or the use of group designs when other 
methodologies (e.g., qualitative, single case experimental designs) may be more appropriate. 
Researchers who implement and report group quantitative designs should explicitly acknowledge 
the internal and external validity threats introduced by less rigorous designs and make clear that 
evidence produced by these designs is only tentative. The Task Force on Statistical Inference 
(Wilkinson et ah, 1999) admonished, “Make clear at the outset what type of study you are doing. 
Do not cloak a study in one guise to try to give it the assumed reputation of another.” 

As noted in the publication Scientific Research in Education (National Research Council, 
2002, p. 97), research design is one aspect of a larger process of rigorous inquiry and the design 
of a study does not itself make it scientific. Other findings reported in the present article permit 
comment about the extent to which broadly defined rigorous inquiry was shown by quantitative 
group design research in El and insights about what steps might be taken to increase rigor in 
future research. 

Measurement Integrity 

Several measurement-related issues emerged across the reviewed studies. Given the central 
role of measurement in rigorous scientific inquiry, these issues deserve particular attention. First, 



ERiC 



184 



Review of Quantitative 1 3 



only 56.6% of the studies reviewed provided information about reliability of scores for primary 
dependent measures and even fewer (20.4%) offered information about validity of scores. 

Second, reliability of scores based on data obtained from the study sample was reported in 
only 1 69 studies. The validity data are more alarming; score validity estimates based on study 
sample data were reported in just 17 studies. 

Third, in over 53% ( n = 242) of the studies reviewed, researchers chose to operationalize 
primary dependent variables using author-developed outcome measures. The use of author- 
developed measures is of concern because many researchers reported no or insufficient 
information about the psychometric integrity of these instruments. 

The reporting practices related to score reliability and validity and the predominant use of 
researcher-developed instruments are not unique to the early intervention literature. Lawson and 
LaGaccia (1990) reviewed 328 research articles published in four special education journals 
between 1985 and 1990. They found that in 66% of the studies the instruments used were author- 
developed. Only 1 8.4% of these studies reported reliability data and 21 .4% reported validity 
data. A recent series of Reliability Generalization (RG) studies has shown that the measurement 
practices found in the present study are consistent with those found throughout the education and 
psychology literatures (Vacha-Haase, Henson, & Caruso, 2002). 

Failure to establish the psychometric integrity of scores derived from dependent measures 
compromises the meaningfulness of the data and limits the substantive conclusions that should 
be drawn about the variables under consideration. Not clear from this review is whether 
researchers failed to evaluate the psychometric integrity of dependent measure scores or merely 
did not report these estimates. 
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Concerns about measurement integrity and measurement reporting practices led the Task 
Force on Statistical Inference (Wilkinson et al., 1999, p. 6) to offer several prescriptive 
guidelines. The Task Force asserted that authors should provide score reliability coefficients for 
the data actually being analyzed, even when the focus of the research is not psychometric. 

It is not sufficient to report reliability coefficients from test manuals. This warrant for 
score integrity is called “reliability induction” and is reasonable if, and only if, (a) the 
compositions, and (b) the score variabilities of the researcher’s sample and the inducted sample 
are both comparable (Vacha-Haase, Kogan, & Thompson, 2000). These inductions require 
explicit and direct comparisons of compositions and score variabilities (Thompson, 2002). 

The Task Force also cautioned that authors who develop their own measures often give 
insufficient attention to the quality of their instruments. They remarked that once a potentially 
defective measure enters the literature, subsequent researchers often are reluctant to change it. 

Guidelines published in the Journal of Early Intervention (Snyder, 2000) offer 
recommendations similar to those provided by the Task Force on Statistical Inference. In 
addition, authors who submit manuscripts to JEI are required to avoid characterizing tests as 
reliable or valid and they must describe the specific type of reliability or validity under 
consideration. When evaluating future research, editors, authors, and consumers should give 
systematic and sustained attention to measurement practices and reporting of psychometric 
evidence to prevent the accumulation of findings and the promulgation of recommended 
practices based on invalid or unreliable measures. 

Treatment Fidelity 

Another facet of rigor in quantitative research is the extent to which the independent 
variable is operationally defined and implemented with fidelity. In 1992, LeLaurin and Wolery 
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proposed research standards for El related to defining, describing, and measuring the 
independent variable. One standard suggested by these authors related to inclusion of measures 
of treatment fidelity (i.e., the integrity with which the independent variable is implemented as 
planned). They suggested that, at a minimum, the operational definition of the intervention 
(independent variable) should be accompanied by descriptions of the implementation schedule 
(e.g., frequency and intensity data). Preferably, treatment fidelity also should be documented by 
measuring the extent of implementation and describing participants’ involvement in intervention. 

Slightly less than one-half of the 450 reviewed studies provided information related to 
intervention implementation schedules. Only 57 studies offered more specific treatment integrity 
data. These findings suggest that readers of research often are left to assume that an intervention 
was implemented or applied consistently. This is a potentially deleterious situation, particularly 
when the research literature is being used to support recommended practices. As LeLaurin and 
Wolery (1992, p. 286) noted, absence of treatment fidelity information may mean that results are 
uninterpretable or that intervention effects cannot be explained satisfactorily. In future studies, 
researchers should produce sufficient documentation related to treatment fidelity and editors and 
reviewers should not recommend publication of articles that do not offer these data. 

Statistical Analyses 

Statistical techniques reported in the reviewed studies were evaluated in relation to five 
general categories of analyses and trends related to these categories were displayed over 9 years. 
Overwhelmingly, univariate parametric procedures were used most frequently across all years. 
Although the use of multivariate parametric methods was expected to increase in later years due 
to the increased availability of statistical packages for personal computers, this trend was not 
evidenced in the studies reviewed. Further, more advanced statistical modeling techniques also 
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did not increase significantly over time. Perhaps more important than the types of statistical 
techniques used to evaluate substantive hypotheses is recognition by researchers of what 
statistically significant test statistics or p values mean and the importance of providing 
“informationally adequate” or sufficient statistics in published research (American Psychological 
Association, 2001). Several recent publications provide guidance about these issues that should 
be helpful in the conduct of future research and in the interpretation and reporting of research 
results (e.g., American Psychological Association 2001; Snyder, 2000; Wilkinson and Task 
Force on Statistical Inference, 1999). 

Magnitude-of-Effect Indices 

As is generally acknowledged (e.g., American Psychological Association, 1994, 2001; 
Wilkinson and Task Force on Statistical Inference, 1999) but not always widely understood (cf. 
McCartney & Rosenthal, 2000; Thompson, 2002), probability values associated with inferential 
statistics do not directly reflect the magnitude of an effect or the strength of a relationship. Thus, 
the Task Force on Statistical Inference recommended magnitude-of-effect indices always should 
be presented for primary outcomes and researchers should add comments that place these effect 
sizes in practical and theoretical contexts. Similarly, the 5 th edition of the Publication Manual of 
the American Psychological Association (American Psychological Association, 2001) states, “. . 

. it is almost always necessary to include some index of effect size or strength of relationship in 
your Results section” (p. 25). The editorial policies of at least 20 journals, including the Journal 
of Early Intervention, now require reporting of magnitude-of-effect indices (Thompson, 2002). 

Only 80 of the studies reviewed reported magnitude-of-effect indices. All reviewed 
studies were conducted, however, before the expectation to report magnitude-of-effect indices 
was widely disseminated. Future reviews of the El research literature should reveal increased 
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reporting and interpretation of magnitude-of-effect indices. McCartney and Rosenthal (2000) 
insightfully propose that these indices should be considered in context and suggest two main 
contexts in which effect sizes should be evaluated: scientific and literature-based. The scientific 
context includes consideration of design, measurement, and method issues that directly impact 
magnitude-of-effect indices. As these authors note, better measurement, design, and method tend 
to produce larger effects. Evaluating effect indices within the literature context suggests that 
researchers should explicitly compare their obtained effects to effects reported in similar 
research. This process is particularly useful for the accumulation of evidence across studies and 
the generation of empirical support for recommended practices. For example, if three or more 
studies concerned with a well-defined and implemented child-focused intervention yield 
comparable effects for certain groups of children, the weight of the evidence in support of a 
recommended practice is enhanced. 

Summary 

The results reported in this paper suggest that the methodological integrity of the 
quantitative research used to inform recommended practices was not uniformly convincing and 
compelling. Nonetheless, these research efforts represent the best-available evidence and offer 
important foundations for future research. As the definition of scientific rigor in El research 
continues to evolve and standards related to ensuring rigor in the conduct and reporting of 
research are refined, the quality of the best-available evidence should be enhanced. In turn, 
confidence about what constitutes recommended practice should advance and contribute further 
to the implementation of evidence-based practices in El. 
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Table 1 

Number and Examples of Independent or Attribute Variables Reported in Studies by 
Recommended Practices Category 



Recommend Practice Category Numbers of Variables Example Variables 



Child-focused 



255 



Family-Based 



68 



Personnel Preparation 



41 



Interdisciplinary Models 



22 



Systems Change/Policy 



11 



Assessment 



10 



Type of Play Context 
Age-of-Start Intervention 
Disability Status 
Maternal Education Level 
Degree of Caregiver Burden 
Type of Parent Training 
Type of Consultation Model 
Professional Discipline 
Type of Training Setting 
Type of Service Model 
Team Model 
Type of Classroom 
Size of Locality 
Type of Placement Criteria 
Type of Agency 
Time of Assessment 
Location of Assessment 



O 



19 5 



Review of Quantitative 24 



Recommend Practice Category 


Numbers of Variables 


Examples of Variables 






Type of Assessment 


Technology 


8 


Augmentative Device Type 
Mode of Communication 
Seating Inclination Angle 



Note. Total number of variables equals 415. 
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Table 2 

Number and Examples of Dependent Measures in Different Categories 



Category 


Frequency 


Example Measures 


Investigator-developed 


242 


Interaction Rating Scale (Fallon & Harris, 1991) 

Survey of transition issues (Shotts, Rosenkoetter, 
Streufert, & Rosenkoetter, 1 994) 

Behavioral State Observation Scale (Guess et al., 1990) 


Commercially-available 


78 


Battelle Developmental Inventory (Newborg, Stock, 



Wnek, & Guidubaldi, & Svinicki, 1994) 

Peabody Developmental Motor Scales (Folio & 

Fewell, 1983) 

Peabody Picture Vocabulary Test - Revised, Form L (Dunn & 
Dunn, 1981) 

Vineland Adaptive Behavior Scales - Survey Form 
(Sparrow, Balia, & Cichetti, D., 1984) 

Published in other research 

or limited circulation 124 Family Needs Survey (Bailey & Simeonsson, 1990) 

Questionnaire on Resources and Stress — Short 
Form (Friedrich, Greenberg, & Cmic, 1983) 

Carolina Record of Infant Behavior 
Family-Focused Intervention Scale (Mahoney, 
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Category 


Frequency 


Example Measures 


Devices 


6 


O'Sullivan, & Dennebaum, 1 990) 

ESCAPE: Eco-behavioral System for Complex 
Assessments of Preschool Environments (Carta, 
Greenwood, & Atwater, 1992) 

Goniometer 


Note. Total number of primary dependent 


measures = 450. 
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Table 3 

Number of Score Reliability and Validity Estimates 



Type of Estimate T otal N umber 


Number Based on Study Sample Data 




Reliability 




Test-retest 


52 


22 


Internal Consistency 


81 


43 


Parallel Forms 


1 


0 


Inter-rater Reliability 


95 


71 


Standard Error of Measurement 


0 


0 


Inter-observer Agreement 


76 


62 




Validity 




Face/Content 


26 


5 


Predictive 


4 


0 


Concurrent 


31 


4 


Construct 


23 


5 


Discriminant 


4 


2 


Social 


1 


1 



Note. More than one type of score reliability or validity could be reported in a single study. 
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Table 5 

Number of Studies Reporting Magnitude-of-EJfect Indices by Category and Type 



Category and Type 


Number of Studies 


Uncorrected variance-accounted-for 


r 2 


35 


R 2 


17 


Wilks’ Lambda 


4 


Eta squared 


2 


Corrected variance-accounted-for 


Adjusted R 2 (Ezekiel, 1930) 


1 


Omega squared 


1 


Mean difference 


Cohen’s d 


5 


Glass’ delta 


4 


Cohen’s k 


1 


Other 


6 



Note. In four studies, magnitude-of-effect values were reported, but the type of index was not 
specified clearly. 
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Abstract 



Descriptive information about the qualitative research studies that were reviewed 
as part of the DEC Recommended Practices project is provided. Seventy-four articles that 
met our criteria were reviewed. These studies were more likely to address issues of 
families, policy, and personnel preparation than other topic areas in early 
intervention/early childhood special education. Most of the studies provided sufficient 
detail in terms of describing the conceptual framework, research strategies and methods, 
and findings. Limitations of the review as well as suggestions for future directions are 
offered. 
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Qualitative research has a long and well-established record in the social sciences. 
Reports from qualitative research are well represented in the general education research 
literature. The use of qualitative research methods is not new to special education and 
disability studies (e.g., Bogdan & Taylor, 1976). However, some have noted that 
qualitative research is not well represented in the major special education journals 
(Bogdan & Lutfiyya, 1992; Reid & Bunsen, 1995). Pugach (2001) wrote recently 
"relative to the larger world of educational research... special education has come to 
accept and undertake the practice of qualitative research methodology only rather 
belatedly (p. 440). 

The use of qualitative research methods appears to be even newer to early 
intervention/early childhood special education. The earliest reports in the Journal of Early 
Intervention that used qualitative methods appeared in 1989 (Kugelmass, 1989; Peck, 
Hayden, Wandschneider, Peterson, & Richarz, 1989; Salisbury, Britzman, & Kang, 1989) 
Previous syntheses of the early intervention literature (e.g., Casto & Mastroprieri, 1986; 
Casto, White, & Taylor, 1983) omit any qualitative studies. In designing a comprehensive 
literature review of effective practices, the Division for Early Childhood (DEC) 
Recommended Practices project (Sandall, McLean, & Smith, 2000) included research 
studies that incorporated qualitative research methods as part of the review of the 
scientific literature. The purpose of this paper is to describe those reports and their 
contribution to recommended practices for the field. 
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Qualitative research is complex, open-ended, and difficult to pin down. However, 
for the purposes of this paper, we begin with the definition offered by Denzin and 
Lincoln, 

Qualitative research is multimethod in focus, involving an interpretive, 
naturalistic approach to its subject matter. This means that qualitative researchers 
study things in their natural settings, attempting to make sense of, or interpret 
phenomena in terms of the meanings people bring to them. Qualitative research 
involves the studied use and collection of a variety of empirical materials... that 
describe routine and problematic moments and meanings in individuals' lives. 
(1994, p. 2) 

Qualitative research has the potential to increase our understanding of children 
with special needs, their families, and those who work for and with them. Qualitative 
research has the potential for increasing our understanding of the intervention issues and 
challenges posed by these children. Context is crucial to the qualitative researcher. The 
researcher needs to understand the context within which the data are produced and 
collected. Understanding the context is key to using the data. Context may mean 
everyday and ritual activities as they naturally occur in, for example, homes and 
classrooms. Further, the notion of context extends to the cultural and sociopolitical 
circumstances in which early intervention/early childhood special education problems 
and issues occur. 

Crabtree and Miller (1992) highlighted two of the distinguishing features of 
qualitative research. First, qualitative research involves multiple methods. Many different 
sampling, data collection, and analytic procedures are possible. For example, among the 
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fundamental methods used for collecting information are interviews, observations, and 
documents or some combination of these. Qualitative researchers choose from the array 
of methods depending on their purposes and questions. Their methods may alter or 
expand during the course of the study as more and more information is collected and 
analyzed. This leads to the second distinguishing feature; qualitative research is recursive 
and cyclical. Collection and analysis usually occur concurrently. And, the qualitative 
researcher returns repeatedly to the data and/or the data source. 

Ferguson and Ferguson (2000) noted that there is no single dimension or "right 
way" to do qualitative research. They argue for thoughtful attention to several dimensions 
of quality. One of the challenges faced by researchers and the reviewers of the research is 
coming to agreement about these dimensions. McWilliam (2000) tackled this challenge 
for the Journal of Early Intervention. He outlined four criteria that are considered 
necessary for submission to the journal. These are: (a) sufficient detail about where the 
investigator is coming from; (b) sufficient detail about what the investigator did and the 
methods employed; (c) sufficient detail about how the investigator (s) arrived at the 
findings; and (d) discussion of the meanings of the findings in relation to theory and 
implications. 

These criteria align with the coding system that was used as part of the DEC 
Recommended Practices project. In this paper we describe the outcomes of the review of 
the research literature in early intervention/early childhood special education (EI/ECSE) 
from 1990 to 1998. The focus of this paper is on those articles that employed qualitative 
research methods. 

Method 
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The DEC Recommended Practices project is described in Sandall, McLean and 
Smith (2000). The methods undertaken to identify the recommended practices are 
described in the article by Smith and colleagues in this issue. 

Procedures 

The literature review was conducted in multiple steps. The articles to be reviewed 
were identified from 48 peer-reviewed journals relevant to early intervention/early 
childhood special education. Project staff reviewed the table of contents. Next, the article 
abstract for each research article was reviewed to determine if the article met the criteria 
for inclusion in the study. The article had to be an investigation of an intervention 
provided to children with disabilities, birth through five and/or their families or those 
who serve them. Articles that passed this screening were categorized by research design: 
group quantitative, single subject design, descriptive/survey design, qualitative design, 
and mixed method design. 

The articles were then reviewed using two coding sheets. Reviewers used coding 
sheet A, a generic coding sheet, across all of the research designs. Coding sheet B, the 
specialized coding sheet, was developed to describe the article according to criteria 
appropriate for that design. 

Specialized coding form 

The specialized coding sheet for the qualitative research reports was developed 
based on review of criteria proposed in textbooks and articles on qualitative research (e.g. 
Denzin & Lincoln, 1994; Lincoln & Guba, 1985; Strauss & Corbin, 1990), and 
discussions with experienced qualitative researchers. The coding form was sent to two 
researchers in EI/ECSE and based on their comments and suggestions, revisions were 
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made and the final coding form produced. The coding form recorded the following 
information: conceptual framework, focus of the study, type of study, description of 
procedures, data sources, length of data collection, description of analysis, evidence of 
credibility, findings in relation to theory, recommended practice(s) supported by the 
study, and the strand or topic area that the practice reflected. 

Coders 

There were seven coders for the qualitative studies. All had experience 
conducting qualitative research. Training began with reading the coding forms and 
supporting materials. Coders then coded one article that was not part of the data set. Next, 
the coder participated in an in-person or telephone conversation with the methodology 
consultant. The coder then coded two or more additional articles that were not part of the 
data set until reaching a level of 85% intercoder agreement with the methodology 
consultant. Coders did not obtain this level of agreement for one item - type of study. 
Originally, coders were given a menu and asked to categorize the study. After lengthy 
conversations about the discrepancies found in coding this item it was determined that 
rather than having the coder determine the type of study (i.e., design or research strategy) 
and place it into a category, the coder simply used the researcher's own words to describe 
type of study. Each article was read and coded by one coder. A second coder read 
approximately one-third of the articles to confirm the recommended practice that was 
supported by the article. 

Results 

Following initial screening of articles, there were 109 articles using qualitative 
methods. Thirty-five were eliminated as they were found not to meet the inclusion criteria 
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or did not support a recommended practice. Thus, there were 74 articles using qualitative 
methods that were included in the final literature review. This represents 9% of the total 
research articles reviewed. When looking at publications by year, we found that there 
were 6 articles published in 1990, 5 in 1991, 4 in 1992 and in 1993, 7 in 1994, 9 in 1995, 
13 in 1996, 14 in 1997, and 12 in 1998. 

Coders also assigned articles to a recommended practice(s) strand. Qualitative 
articles most often supported the family-based (24), policies/procedures/systems change 
(22) and personnel preparation (22) strands. Articles supported other strands less often: 
child-focused (12), interdisciplinary (7), and technology (2). No qualitative studies were 
assigned to the assessment strand. 

Coders recorded whether or not the researcher described the conceptual 
framework for the study. Conceptual framework meant that the problem or area of study 
was placed within a body of theory and practice. Forty-four of the 74 articles (59%) 
provided an adequate description of the conceptual framework. 

Qualitative studies may use any number of designs or strategies, methods for 
collecting information and methods for selecting participants. The study may, in fact, 
evolve over time. In coding the studies, reviewers looked for detail in the desciption of 
the methods and procedures used. 

Among the varied strategies of inquiry (or design) that a researcher might use are 
ethnography, case study, and biography. As noted earlier, coders found it difficult to 
record reliably the type of strategy when using predetermined categories. Using the 
researcher's own words, we found that the majority of researchers (41) described their 
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study as qualitative (55%). Eighteen studies (24%) were called an ethnography. The 
remaining articles were dispersed among several other research strategies. 

The qualitative researcher has several methods for collecting information. 
Observations were used in twenty-six studies (35%). These were usually combined with 
field notes. Interviews of various types (e.g., individual, group, formal, unstructured) 
were used in sixty-five studies (88%). Twenty studies (27%) reported the use of 
documents and artifacts. Sixteen studies (22%) used other methods. In thirty-two of the 
studies (43%), some combination of data collection method was used. 

In that a variety of methods can be used for collecting information and for 
determining from whom information is collected, we wanted to know if researchers 
provided careful, detailed description of their methods. Coders found that 59 studies 
(80%) provided detailed description of the procedures used to select participants. Sixty- 
one (82%) provided detailed description of the data collection procedures. 

The researchers reported a variety of techniques for determining the 
trustworthiness of the data. Trustworthiness refers to four major concerns: truth value, 
applicability, consistency, and neutrality (Lincoln & Guba, 1985). Techniques reported 
included member checks (26), multiple methods (24), intercoder reliability checks (22), 
multiple sources (19), keeping an audit trail (5), and other (34). Forty of the reports 
(54%) used more than one technique. 

An indicator of the credibility of the data has to do with whether or not the 
researcher has engaged with the participant and/or site for a sufficient period of time. 
Fifty-six studies (76%) reported the length of the data collection period. This ranged 
from a single interview to three years of contact with the participants. 
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Qualitative researchers arrive at their reported findings through construction of 
their interpretation. Lincoln and Denzin (1994) wrote that interpretive practice is both 
artful and political. McWilliam (2000) recommended that reviewers look for sufficient 
detail in terms of a thoughtful, informed process. In the articles reviewed, researchers 
described their analytic procedures in varying levels of detail and in varying ways. Some, 
for example, provided charts that detailed the analytic steps. Coders found that 51 of the 
articles provided detailed description, 61 were judged to have used an appropriate 
analytic method, and 40 were judged to be inductive. Thirty-four of the articles (46%) 
were coded as detailed, appropriate, and inductive. 

Finally, coders assessed whether or not the study provided a discussion of the 
meaning of the findings in relation to theory. That is, coders looked for an explanation of 
how the findings advanced theory. Thirty-eight studies (51%) were found to have done 
so. 

Discussion 

The purpose of this paper was to provide more information about the qualitative 
studies that were reviewed for the DEC Recommended Practices project. The literature 
review encompassed the period from 1990 to 1998. During that time period qualitative 
research designs were used rarely. This finding is consistent with previous reports of the 
special education literature (Reid & Bunsen, 1995). However, it appears that there was an 
increase in the use of qualitative research methods in early intervention/early childhood 
special education during the time period studied. 
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For the most part, the qualitative studies centered on issues and questions of 
policies, families, and personnel preparation. Qualitative methods were not used across 
all of the strand or topic areas. 

Most of the studies reviewed provided adequate description of the primary 
indicators of quality of qualitative studies. On items related to description of the 
conceptual framework, selection of participants, data collection methods, indicators of 
trustworthiness, analytic steps, and relation to theory, 50% or more of the studies were 
coded as providing sufficient detail. That means, that there were published studies that 
did not adequately address these indicators. 

There was more difficulty in providing a careful description of the design or 
nature of the inquiry. This posed a problem for establishing reliability but when we 
recorded the researcher's words we found that most simply described their study as a 
"qualitative study". Qualitative researchers and those who write about qualitative 
research come from a variety of different traditions, perspectives, disciplines, and 
historical time periods. This may explain why the terminology is sometimes confusing. It 
may also explain why most researchers whose studies were reviewed for this project 
elected to use the most general term to describe their research strategy. 

This report has a number of limitations. The articles reviewed were drawn from a 
pool of 48 peer-reviewed journals. Thus, articles appearing in journals not on our list 
were not reviewed. Further, qualitative studies lend themselves to longer, richer 
narratives than are possible in many journals. Books and chapters were not reviewed so 
again, important papers and findings may have been missed. 
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Researchers may use qualitative methods while not embracing fully an 
interpretivist approach to their thinking about and doing research. In this project we did 
not examine the researcher's point of view but rather accepted studies that used generally 
accepted qualitative methods as fulfilling our requirement as qualitative research. 

Qualitative studies may confirm or test theory. More likely, qualitative studies are 
formulated in order to describe phenomena or to discover or generate theory. Therefore, 
qualitative studies that appeared in the literature during this time period but did not 
investigate an intervention were not included in the review. This another reason why this 
report cannot be considered a comprehensive review of the qualitative research in early 
intervention/early childhood special education. The current review does contribute to our 
understanding of the use of qualitative research methods to address questions of EI/ECSE 
practice during the time period of 1990 to 1998. 

The findings do suggest some next steps and future directions. First, a larger 
review could be undertaken that examines all of the qualitative studies in EI/ECSE that 
were reported in the 1990s and to the present. Such a review could provide a better 
picture of how and when these methods are used. The current review found that 
qualitative methods were most often used to study issues related to families, policies, and 
personnel preparation. A deeper analysis of these studies would offer a better 
, understanding of the kinds of questions that were asked and the kinds of interpretations 
that were derived from the information collected. Qualitative research is well suited to 
providing a "voice" for the various stakeholders in early intervention/early childhood 
special education by listening to their voices and considering their voices within the full 
context of their experiences. Qualitative research can broaden and deepen our 
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understanding of the experience of disability for young children, family members, 
teachers, and therapists. Qualitative research can extend our understanding of the ways 
that services are organized and provided. This research approach can illuminate 
problems and issues in early intervention/early childhood special education and 
potentially help to transform practice. 

Qualitative studies accounted for a relatively small portion of the published 
literature and consequently, the literature reviewed for the Recommended Practices 
project. From 1990 to 1998 we saw an increase in the number of published studies using 
qualitative methods. The influence of qualitative research on the practice of early 
intervention/early childhood special education is a story that is yet to be written. 
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Abstract 

The most recent compilation of DEC Recommended Practices in Early 
Intervention/Early Childhood Special Education (Sandall, McLean, & Smith, 2000) 
contains 240 recommended practices across 7 strands. The recommended practices 
evolved from a process involving input from literature reviews, scientific experts, 9 
stakeholder focus groups, and field validation of the synthesized practices (Sandall, 
McLean, & Smith, 2000). One of the 7 strands addresses recommended practices in 
interdisciplinary models. The 19 practices in this strand emphasize teamwork, loose 
boundaries between disciplines, functional intervention, and support to caregivers. The 
article describes the process used to identify these 19 recommended practices by focusing 
on the literature base of 30 articles that supports the interdisciplinary models strand. An 
analysis of the literature will be used to understand the types of research (e g., qualitative 
or quantitative) and the elements (e g., study setting, study participants) used to support 
particular practices. The article will also suggest how these recommended practices can 
become the foundation by which practitioners in early intervention/early childhood 
special education go about their daily routines and practices. By examining the empirical 
base in the published literature, it is possible to suggest modifications that could help a 
team move towards meeting recommended practices and suggest future research 
endeavors to strengthen the empirical support available in this field. 
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The Development of Recommended Practices 

Many professions view the development of recommended practices as a means by 
which they can provide information and support to their constituents while attempting to 
insure that a level of best practice is achieved. Documents containing best practices have 
been developed by a variety of sources and come in the form of guides, standards, and 
position statements. Within the early childhood field, descriptions of what adults should 
do with young children has been a staple of the literature at least since 1986. It was 
during that time that the term “developmentally appropriate practices” began to be used 
to describe programs serving children from birth to age 8 (National Association for the 
Education of Young Children, 1986a) and for 4- and 5-year-olds (National Association 
for the Education of Young Children, 1986b). By 1990, developmentally appropriate 
practices and inappropriate behaviors were being measured in observational studies 
(Burts, Hart, Charlesworth, and Kirk, 1990; Hyson, Hirsch, & Rescorla, 1990), followed 
by definitions of best practices in kindergarten (Bryant, Clifford, & Peisner, 1991; 
Zepeda, 1993). By 1993, the effectiveness of practices labeled developmentally 
appropriate were debated in early childhood special education circles (Carta, Atwater, 
Schwartz, & McConnell, 1994; Guralnick, 1993; Johnson & Johnson, 1993). 

More recently, the National Association for the Education of Young Children 
(NAEYC) has published position statements on developmentally appropriate practice 
(DAP) for educating and caring for young children (Bredekamp & Copple, 1997). 
Another contribution to guide professionals in their practice with young children comes 
from the Head Start Performance Standards (Administration on Children, Youth, and 
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Families, 1999). When looking more specifically at practices aimed at the education or 
early intervention of young children with disabilities, there are the DEC Recommended 
Practices (Sandall, McLean, & Smith, 2000). These recommended practices “are 
intended to help meet the individual and unique needs of young children with disabilities 
and their families” (Sandall, McLean, & Smith, 2000, p. 10). 

Other related professional organizations and fields also have documents that 
provide a resource to their membership regarding the nature of the practice. The Guide to 
Physical Therapist Practice (APT A, 2001) is an example of a product developed within 
and for a specific profession by a professional organization. It identifies common 
features of patient/client management, but it does not provide specific protocols for 
treatments or clinical guidelines. Nor does it require that a physical therapist adhere to its 
recommendations in order to practice as a licensed allied health professional. Volume I 
was published in 1995, and Volume II followed in 1997. These two volumes were later 
joined and renamed Part One and Part Two of the Guide to Physical Therapist Practice. 
and the second edition of both parts resulted in the most recent version published in 2001 
(APTA, 2001). 

DEC initiated an effort in 1991 to summarize the early intervention/early 
childhood special education (EI/ECSE) knowledge base. These efforts resulted in a list 
of 405 recommended practices first published in 1993 (DEC Task Force on 
Recommended Practices). These practices were derived from stakeholder focus groups 
and validated by a national sample of 500 individuals, but the process did not include a 
systematic review of the literature. In 1998, DEC leaders secured a federal grant to 
update and revise the 1993 list of recommended practices. 
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Not only were the practices updated, but a more rigorous process was used to 
carefully develop a set of recommended practices that could be implemented at the child, 
family, and systems levels. Much of the additional rigor used in the revision of DEC’S 
recommended practices was based on the commitment by DEC to include a review of the 
literature as a basis upon which to build from the empirical research into practice. 

The Methods Used to Update the Recommended Practices 

Recommended practices in EI/ECSE emerged after an arduous process involving input 
from literature reviews, scientific experts, and stakeholder focus groups (Sandall, 

McLean, & Smith, 2000). Although the focus of this article is the literature base for the 
interdisciplinary-models practices, it is important to understand the process through 
which the recommended practices were developed and validated. The process began with 
scientific expert and stakeholder (parents, practitioners, administrators) focus groups 
meeting to develop a list of suggested practices. Simultaneously, investigators began an 
examination of empirically based research published in peer reviewed journals from 1990 
through 1998. Individual coders read articles and completed two coding sheets for each 
article. A generic coding sheet was used uniformly across all articles. This coding sheet 
recorded the following information: article title, authors, journal, year, general 
description of participants, gender of participants, mean age, ethnicity, disability, severity 
level of disability, educational level of family members, income level of family members, 
and study setting. A specialized coding sheet designed specifically to highlight 
additional information about research rigor based on the method used in the articles was 
also completed. The specialized coding sheets were used to insure that each article was 
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appropriately reviewed according to the research method used in the article (Smith, 

Strain, Snyder, Sandall, McLean, Kamey & Sumi, submitted for publication). 

The results of the coding process were verified by intercoder agreement across the 
group of 42 first-stage coders and by 29 additional coders who independently read and 
coded approximately one third of the articles. In addition to this review and examination 
of the empirical literature base, stakeholder focus groups were used to further define 
practices in each strand. Finally, a national field validation was used to confirm the list 
of 250 practices supported by both literature and focus groups. Criteria that queried both 
the respondents’ agreement with the concept of each item as a recommended practice and 
the frequency with which the item occurred resulted in all 250 practices being validated 
(McLean, Smith & Sandall, submitted for publication). A final review by project 
investigators to eliminate redundancy resulted in a list of 240 recommended practices. 

Thus, the process resulted in 240 practices representing seven different strands 
that had been selected prior to the identification of the practices. One of these seven 
strands of recommended practices is the one on “interdisciplinary models”. This article 
summarizes the literature only from the interdisciplinary models strand. The 
interdisciplinary strand was organized into practices that support working together, cross- 
disciplinary boundaries, focus on function, and that highlight regular caregivers and 
routines (McWilliam, 2000). These four themes emerged from the expert focus group and 
were supported, to a large extent, in the literature. 

Table 1 shows the 19 recommended practices and the groups or the literature that 
supported each practice. In addition, the table provides information on some 
characteristics of the studies reported in the literature and identified as supporting an 
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interdisciplinary practice. The interdisciplinary strand consists of 19 recommended 
practices suggested and supported by three stakeholder groups: practitioners, 
administrators, and families. An independent search for practices falling into this strand, 
or category, (interdisciplinary models) was conducted. Many of these interdisciplinary 
recommended practices were supported through a literature base of 30 articles that had 
been reviewed and published in peer-reviewed journals. The practices in the literature 
were matched against the practices identified and confirmed by experts and stakeholders. 
Twelve of the 19 recommended practices have one or more references in this body of 
literature. Four have one reference, 4 have two references, and 4 have more than two 
references. Five of the practices are not supported by literature or by a stakeholder group 
but were determined by the expert focus group and project investigatorsto be included as 
recommended practices in EI/ECSE. Two practices were identified early in the process 
by a stakeholder group as a part of the Learning Environments Strand. The practices that 
emanated from the Learning Environments Strand were eventually subsumed into one of 
the other strands of recommended practices later in the process. None of literature-based 
interdisciplinary practices was dropped. 



Insert Table 1 here 



The 30 articles represent several different types of study designs. As Table 1 
shows, by far the most common design was a group design (N=60%), but a considerable 
proportion of the studies used qualitative methods (N=23%). Only a few (5 studies or 
17%) used single-subject (N=2), survey (N=2), or mixed methods (N=l). The study 
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settings included children’s homes (N=9), segregated classrooms (N=5), integrated or 
inclusive classrooms (N=5), hospital or clinic-based settings (N=4), and other settings in 
the community (N=l 1). The study participants consisted of personnel (N=12), children 
with disabilities (N=15), typically deveoping children (N=3), families of children with 
disabilities (N=l 1), and families of typically developing children (N=l). Several of the 
studies were executed in more than one location (e.g., home and segregated class) and 
included more than one type of study participants (e.g., families and personnel). In 
addition, many of the studies (N=9) included multiple disability types or diagnoses across 
the study participants. These often included cerebral palsy, developmental delay, mental 
retardation, speech and language impairment, and behavior disorders. Several studies 
focused on a more specific group by including only participants from one or two 
identified diagnoses such as children with sensory impairments (vision and hearing) 
(N=2), developmental delay (N=3), or Down syndrome (N=l). 



Interdisciplinary Models 

Before exploring these recommended practices in greater detail, it is important to 
understand the meaning and the use of the term interdisciplinary models. The terms 
interdisciplinary and transdisciplinary are often confused and, on occasion, are used 
interchangeably. Although the two are not necessarily synonymous, they both represent 
the concept of persons from multiple professional backgrounds and expertise working 
together towards assisting the child and family to achieve one set of goals and objectives 
or outcomes. Thus, whereas interdisciplinary refers to the fact that there is interaction 
among professionals from different disciplines, transdisciplinary refers to a specific way 
in which those interactions occur (McWilliam, 1 996). Although leaders in early 
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intervention are strongly in favor of the transdisciplinary approach (see Harbin, 

McWilliam, & Gallagher, 2000), it is viewed with caution among some practitioners 
from specialized disciplines. The DEC therefore chose the broader term as being more 
inclusive of the concept of working together with others. Despite these differences in 
terminology, the interdisciplinary recommended practices clearly convey the notion that 
EI/ECSE should be provided by a team that works together. 

To understand the interdisciplinary recommended practices better, it is helpful to 
examine the literature base of 30 articles from which many of these practices were 
generated and supported. The articles were published in 18 journals with the largest 
groups of articles (each n = 4 or 13%) coming from The American Journal of 
Occupational Therapy and Topics in Early Childhood Special Education. Two other 
journals, the Journal of Early Intervention and Pediatric Physical Therapy, were a source 
for three articles or 1 0% of the literature base each. Thus, almost half of the total 
literature base for the interdisciplinary recommended practices is published in only four 
journals. 

Looking at Specific Recommended Practices and Their Literature Base 

Slightly over 30% of the practices in the interdisciplinary strand were not 
confirmed by any empirical literature. These six practices were identified by the expert 
focus group or by one of the stakeholder groups and later validated by the field during the 
process described in an earlier section of this article. The practices span several different 
themes and cannot be linked to any one area of policy, intervention, or skill. Their 
commonality arises from the understanding that all are important in the delivery of early 
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intervention services or early childhood special education through an interdisciplinary 
model or array of services. 

Four practices in the interdisciplinary models strand were recommended by the 
scientific expert group but not supported by the literature or by a stakeholder group: 

□ Team members are knowledgeable about funding and reimbursement policies and 
advocate for policies that support recommended practices; 

□ Team members use a transdisciplinary model to plan and deliver interventions; 

□ Team members select child and family priorities for intervention based on child and 
family functioning (not services) and determine what is interfering with growth or 
progress in each priority area; and 

□ Team members use activities within the range of current functioning (i.e., 
individually appropriate activities). 

In addition to these five practices, three additional interdisciplinary recommended 
practices were supported only by a single stakeholder group (either Practitioner 
Stakeholder Group or Learning Environments Focus Group) but not by any empirical 
literature: 

All team members participate in the IEP/IFSP process; 

□ Team members plan to provide services and conduct interventions in natural learning 
environments; and 

□ Team members focus on the between-sessions time (i.e., build in activities that can be 
carried out across time and contexts). 



O 

ERIC 



227 



Practices Across Disciplines in Early Intervention: The Research Base 1 1 



Why the Field Should Move Ahead With a Limited Literature Base 

Despite the weak literature base available to support the 1 9 interdisciplinary 
recommended practices, it is essential that the field of EI/ECSE move ahead to 
implement the practices in home and community based settings. Nothing would be 
gained by failing to recognize the value of a recommended practice that has been lauded 
by experts and practitioners in the field simply because the current literature base fails to 
contain sufficient evidence that addresses the particular practice. The rate of published 
research to guide practice is lower than the rate at which practices are actually needed to 
meet the needs of children and families. The early intervention field across disciplines 
has had a mixed history of tying together research and practices. Practices range from 
those that are proven to those that are unproven and from those that are commonly 
employed to those that are uncommon. Figure 1 shows these two dimensions of research 
support and the frequency of use of early intervention practices recommended by the 
field. Four examples have been drawn from the interdisciplinary models practices, to 
show how each of these practices can be evaluated using the two-dimensional model in 
Figure 1 . Because we do not have the data on usage of each practice, we cannot 
accurately place any of the 1 9 interdisciplinary recommended practices across the two 
dimensions. However, individual practitioners and administrators can estimate the 
frequency with which each practice is found in their community or setting and can 
examine Table 1 to determine the extent to which a practice is supported by research. 

A number of practices and positions in early childhood have moved forward with 
only limited empirical data supporting them. Inclusion is an example of such a practice. 
Particularly in its earlier days, the concept of inclusive classrooms was recommended and 
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encouraged long before there was a literature base to provide conclusive support of this 
practice. There are a number of therapeutic interventions implemented by related service 
personnel (e.g., occupational therapists and physical therapists) that currently are limited 
in acceptance by a lack of reported evidence or a strong research base to support them. 
Even so, many of these interventions are used effectively, according to therapists reports, 
on a daily basis with young children in home, school, community, hospital, and clinic 
based settings. 

Just as the health care system is now requiring greater accountability through 
evidence-based practice, EI/ECSE practitioners could conduct action research, for 
example, and build efficacy through data collection as part of daily practice 
(methodological rigor notwithstanding). Without the ability to demonstrate successful 
outcomes that have been measured and recorded through such research, a sense of 
skepticism looms over the ability of the field to demonstrate that it is knowledgeable and 
progressive. Along the same vein, it is important to examine objectively any 
questionable to insure that the field moves forward with practices that truly represent 
what we know to be the best. 

It is important that all members of the early intervention team not lose sight of the 
need for empirical support as a means of supporting the practices that we assume to be 
the best for children and families. Although observation, experience, and shared 
interactions all support some of the practices we believe to be the best, there continues to 
be a need for an empirical base to provide a strong foundation. 
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The Practices 

Teamwork and Family Decision Making 

Because of the perceptions of stakeholders, and as a result of the literature, DEC 

has recommended 19 practices for providing early intervention across disciplines. The 
first six practices deal with teams including family members' making decisions and 
working together (see Table 1). The underlying themes of these practices are that 
professionals should work in harmony (not at cross purposes or in a hierarchical manner) 
and that natural caregivers should be supported (not usurped by a service-driven system 
or expert-model consultation). The role of families in these practices reflects a "collective 
empowerment model" (Turnbull, Turbiville, & Turnbull, 2000, p. 641), in which both 
professionals and families achieve mastery over their resources to reach their desired 
goals. Tumbull et al. discuss the "synergistic power" (p. 642) that is essential to this 
model and make the following six assumptions: (a) centrality of the family, (b) decision 
making based on family choice, (c) focusing on family strengths, (d) access to resources, 
(e) family participation in the process, and (f) changing the community ecology. In 
general, early interventionists across disciplines are encouraged to work together and 
with families not just in a new way but with a new way of thinking. 

Transdisciplinary Service Delivery 

The next grouping of practices involve crossing disciplinary boundaries. Bruder 
(1997) has discussed the importance of transdisciplinary service delivery in the context of 
curricula for children with disabilities: 

"Interventions from specific therapeutic disciplines can result in a child learning 
isolated skills that have minimal relevance to the remainder of the intervention 
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program. ... As second-generation research on early childhood intervention 
expands, individual therapeutic interventions must be encompassed within a 
larger curriculum framework that identifies all teaching (e.g., including therapy) 
episodes." (p. 538) 

In home-based services, especially, professionals' willingness to engage in role release 
and role acceptance (Practice 7) and to use a transdisciplinary model to plan and deliver 
interventions (Practice 8) can result in several fundamental shifts in the way early 
intervention is carried out. First, engaging in role release will allow professionals from 
different disciplines to add consultation expertise to their existing skills (Hanft & Place, 
1996). Second, they will approach children and families from a holistic viewpoint rather 
than holding fast to a domain- or discipline-specific approach, thereby allowing 
collaborative problem solving to occur (Utley & Rapport, 2000). Third, they will be 
engaged in ongoing staff development as they learn from their colleagues (through role 
acceptance). Although transdisciplinary service delivery (Chapman & Ware, 1999; 
Lamorey & Ryan, 1998) is threatening to many professionals, especially when they have 
only passing familiarity with the idea, teams using it have reported positive effects on 
their personal and team development (Kaczmarek, Pennington, & Goldstein, 2000). 
Focus on Function 

Eight additional interdisciplinary practices deal with intervention focused on 
function, not services. For example, a service focus might involve prescribed "oral- 
motor" activities to increase a child's oral-motor tone but focuses instead on the service 
necessary to increase tone. This is considered a service focus because the intervention 
does not address why a child needs to have better oral-motor tone. Furthermore, the 
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specific treatment activity describes or implies a singular approach that might or might 
not fit into the child's ecology. In contrast, a function focus might involve teaching the 
child to chew so he can eat more solid food. Or it might involve teaching the child to use 
two-syllable words to label objects and people so he can better express his needs and 
desires. Note that both of these functions (i.e., chewing and speaking) could have been 
the reason or outcome for the "oral-motor" activities. A service focus does not readily 
identify the functional outcome, whereas a function focus — by definition — does. Another 
important point of a function focus is that interventions tend to be (a) appropriate for 
regular routines, (b) able to be carried out by regular caregivers (versus specialists only), 
and therefore (c) are naturally delivered at a higher rate (multiple times a day) than 
service-focused interventions that are directly linked to intervention provided by a 
specialist (e g., once a week). 

The following checklist identifies characteristics embedded in the 
interdisciplinary recommended practices listed in Table 1: 

1 . Identify the contexts in which the child lives (e.g., routines, activity settings) and 
use these to organize assessment; 

2. Change where, when, and how the child receives intervention, as necessary; 

3. Make interventions natural to the child’s and family’s life; 

4. Whenever possible, seek reimbursement for “indirect” services, especially 
consultation — and definitely consultation to natural caregivers, at the same rate as 
“direct” services, since the value of this service may ultimately be just as 
significant as the therapist providing direct intervention to the child. 
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5. Use the least-to-most principle about intrusion, assistance, or prompting with 
children and families (Wolery, Ault, & Doyle, 1992). 

Caregiver-Mediated Intervention 

The final three interdisciplinary practices indicate a key principle in a modem, 
functional approach to working across disciplines: intervention for the child depends on 
what occurs between professional sessions (McWilliam, 2000; McWilliam & Scott, 

2001). That is, children learn skills and make developmental gains as a result of repeated 
interactions with their environment, dispersed over time, not in massed trials once or 
twice a week. The concept of "integrated instruction and therapy" (p. 514) has been 
described as a key factor influencing the implementation of environmental arrangements 
and strategies for teaching preschoolers with disabilities (McWilliam, Wolery, & Odom, 
2001). Current thinking even from single disciplines, as in this case (speech-language), 
includes an understanding that (a) natural environments are suitable intervention contexts 
from a social-interactive language theory perspective, (b) consultative approaches allow 
language to be seen in its social context, (c) in-class models are better than pull-out for 
generalization of skills, and (d) pull out services for intervention should be used 
thoughtfully if at all (Paul-Brown & Caperton, 2001). Because many of the services 
represented by the different disciplines in early intervention are provided intermittently, 
recommended practices state that those sessions focus on supporting regular caregivers 
(i.e., the people who spend enough time with the child to make a difference) to carry out 
effective interventions. This fundamentally changes the role of some of these services, 
which might have been based on a model more appropriate for older children or even 
adults — a model based on the client’s absorbing the massed-trial information in a session 
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and then transferring it to real-life situations. In early intervention, an adult is needed to 
effect that transfer, because young children learn in context — not in isolation of context 
(Dunst, Hamby, Trivette, Raab, & Bruder, 2000). 

In no way does this diminish the importance of the individual disciplines. In fact, 
the functional model discussed here recognizes that professionals from the various 
disciplines have expertise that is very much needed by regular caregivers. Some of the 
challenges for professionals as they come to understand this modus operandi are that (a) 
intervention suggestions need to be customized to the individual caregiver, who might 
not carry out the interventions the way the professional would if the professional lived 
with the child; (b) professionals will need to give credit for child progress to caregivers 
(or at least share it with them); and (c) hands-on work with the child (which will 
continue) will be limited more specifically to demonstration, assessment, and showing 
the caregiver that the professional loves the child; seldom will it be to teach the child 
directly. These challenges are not insignificant, and some professionals overgeneralize 
these ideas to conclude that the functional approach means their discipline is not valued, 
they will never work with the child again, or they have assumed the role of a social 
worker. Of course, this is not true, but it does show that an understanding about how 
children learn and how intermittent services can help them does not come easily to many 
professionals isolated within the training and structure of their own discipline. 
Conclusion 

The professional who uses these interdisciplinary recommended practices 
successfully has to know child functioning and intervention strategies, family and 
classroom functioning, and collaborative consultation skills. When professionals have 
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this wide array of knowledge and skills, they are respected and highly valued by their 
colleagues from other disciplines and they add to the outcome of effective and efficient 
early intervention for young children with disabilities and their families. 
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Figure 1 . Dimensions of research support and frequency of use for early intervention 
practices, with examples from interdisciplinary models. 
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Abstract 

Results are reported from the field validation of the DEC Recommended Practices 
as part of a national effort by the Division for Early Childhood (DEC) of the Council for 
Exceptional Children (CEC) to develop evidence-based practices for Early 
Intervention/Early Childhood Special Education. Validation surveys were received from 
388 individuals including practitioners, parents and administration/higher education 
personnel. All of the practices were supported as recommended practice by the 
respondents. Additional information was obtained from the respondents about the extent 
of current use of the practices. 
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The DEC Recommended Practices In Early Intervention/Early Childhood Special 

Education: Field Validation 

In the fall of 1 998, DEC and several university partners were awarded a federal 
grant to update and revise the original DEC Recommended Practices that were developed 
in 1993 (DEC Task Force on Recommended Practices, 1993). The procedures for 
accomplishing this task were completed in three stages. The first stage involved the 
generation of recommended practices. This stage was accomplished by analyzing the 
empirical research literature and conducting focus groups with key stakeholders in the 
field including content experts, family members, practitioners, and administrators. The 
second stage involved the synthesis of practices identified by focus groups with those 
identified from the literature. The procedures followed in stages one and two are 
described in more detail in Smith, McLean, Sandall, Snyder, and Broudy (2000). The 
third stage of this process involved the field validation of the practices, which is the focus 
of this paper. 

The purpose of this field validation was to obtain ratings from key stakeholders in 
the field relative to the validity of the practices which emerged from the review of the 
literature and the focus groups. A national validation was conducted for the original 
DEC Recommended Practices (Odom and McLean, 1996). The results of that field 
validation served as a measure of social validity, and also measured the extent to which 
respondents had experienced the recommended practices in programs with which they 
were familiar. The field validation for the revised set of DEC Recommended Practices 
was designed to serve the same two purposes. 




254 



4 



Assessing the social validity of practices which are being recommended to 
members of a professional discipline is a means of evaluating the acceptability of the 
practices to the constituent stakeholders (Wolfe, 1978) and thus also the extent to which 
the practices are likely to be implemented by the stakeholders (Baer, 1986). The overall 
importance of measuring social validity was emphasized by Peters and Heron (1993) who 
proposed social validity as one of five criterion to be used when identifying best 
practices; other criterion included a sound theoretical base, convincing and compelling 
methodology and design, consensus with the literature and production of desired 
outcomes. Social validity was assessed by Meyer, Eichinger and Park-Lee (1987) in their 
work to identify and validate program quality indicators for students with severe 
disabilities. Meyer et al. included groups of participants in their validation who 
represented different constituencies in the field. A comparison of these groups revealed 
significant group differences in relation to the perceived value of the program quality 
indicators. 

The procedures used in the field validation of the revised DEC recommended 
practices were similar to those followed by Meyer et al. (1987) and to those followed in 
the validation of the original DEC recommended practices (Odom, McLean, Johnson, & 
LaMontagne, 1995; Odom & McLean, 1996). Two major questions were posed: (1) To 
what extent do key stakeholders in early intervention agree that practices generated 
through the literature review and focus groups should be recommended practices for the 
field?, and (2) To what extent do key stakeholders report that practices generated through 
the literature review and focus groups are used in programs with which they are familiar? 
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Method 

Respondents 

Practitioners, parents, and higher education professionals/administrators were 
selected as the three stakeholder groups to comprise the validation sample. Recruitment 
of stakeholders was accomplished in several ways. First, volunteers who represented the 
various stakeholder groups were solicited at the annual DEC conference and through a 
notice published in Young Exceptional Children. The DEC Family Consortium 
Committee also assisted with the identification of parents who might be willing to 
complete a validation questionnaire. A total of 21 1 individuals volunteered: 78 
practitioners, 62 from higher education, 46 administrators and 25 parents. The targeted 
numbers for each of these stakeholder groups was consistent with the numbers from the 
original validation of DEC recommended practices: 400 practitioners, 200 from higher 
education/administration, and 200 parents. The greater number of practitioners identified 
was a reflection of the desire to have those who would be implementing the practices 
strongly represented. To get a total of 400 practitioners, 322 names were randomly 
selected from the DEC membership list to add to the 78 volunteers. To complete the 
higher education/administration group 92 names were selected from a list of currently 
funded personnel preparation projects and a list of Part C/619 coordinators for the states 
and territories. To complete the parents group, 175 names were randomly selected from 
the mailing list of the Federation of Children with Special Needs. A cross check was 
completed with the lists of individuals who had participated in the first two stages of this 
project so that those individuals involved in the development of the practices were not 
included in the validation. 
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Validation Questionnaire 

A questionnaire was developed for the field validation that asked respondents to 
rate each practice according to the following two questions; (1) Is this a recommended 
practice?, and (2) To what extent is this practice followed by programs with which you 
are familiar? Due to the large number of recommended practices generated from the 
literature review and focus group synthesis (n=250), two forms of the validation 
questionnaire were created by assigning odd-numbered items to Form A and even- 
numbered items to Form B. Each respondent received either Form A or Form B with 125 
practices to validate. On each form, the practices were organized by strand. 

Figure 1 includes a sample of the first three items in the Assessment strand from 
Form B. Respondents used a Likert-type scale with ratings of strongly agree (1), 
agree (2), disagree (3), and strongly disagree (4) or undecided (?) to indicate whether the 
practice should be considered as a recommended practice. A Likert-type scale was also 
used to rate current use with categories of frequently (1), sometimes (2), rarely (3), 
never (4) and undecided (?). 

Procedures 

Participants were mailed a validation questionnaire and a letter that asked them to 
return the questionnaire unanswered if they thought they did not have the background or 
experience to respond to the items or if they could not respond within 1 month. The 
initial mailing to respondents was followed 1 month later by a postcard reminder with a 
second postcard reminder mailed approximately 1 month after the first. 
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Results 

Thirty-eight of the 800 surveys mailed were returned unanswered; yielding an 
adjusted sample size of 762. Based on this number, a return rate of 5 1% (n = 388) was 
obtained (21 1 practitioners, 55 parents, and 122 higher education/administrators). The 
mean years of experience for respondents was 14.23 years with a range from .5 year to 39 
years. Of the respondents, 91 characterized themselves as primarily involved in 0-3 
services, 107 characterized themselves as primarily preschool, 122 identified themselves 
with a 0-5 age range and 68 indicated “other” or did not indicate any particular age 
group. 

Recommended practice 

The criterion used to determine whether a practice should be considered to be a 
validated recommended practice was that more than 50% of the respondents to a 
particular item indicated strongly agree or agree in response to the statement “This is a 
recommended practice.” All 250 practices included in Form A and Form B of the survey 
met this criterion. Table 1 lists the frequency of responses to this first question on the 
questionnaire. These data have been aggregated by strand for presentation. 

Current Use 

Table 2 shows the results of the current use rating on the questionnaire, 
aggregated by strand. Ratings of frequently ranged from a low of 17% (Technology) to a 
high of 32% (Assessment). Ratings of sometimes ranged from 38% (Assessment, Policy 
and Systems Change, and Personnel) to 50% (Family-Based). Ratings of rarely ranged 
from 17% (Child-Focused) to 26% (Technology). Ratings of never ranged from 2% 
(Family-Based and Interdisciplinary) to 5% (Technology). Missing data included items 
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left blank (not rated) and those where undecided (?) was circled. Missing data ranged 
from 5% (Assessment and Family-Based) to 16% (Personnel). 

Comparison of respondent groups 

Mean ratings were computed for practitioners, parents and higher 
education/administration personnel for each strand to determine whether differences in 
ratings existed across the groups. Separate, one-way ANOVA’s were conducted to 
examine differences. The p critical value was adjusted according to the Bonferroni 
formula to adjust for experiment-wise error; the adjusted value was .007. Tukey post 
hoc analyses were conducted to determine the direction of the difference when a 
significant main effect occurred. 

Table 3 shows the ANOVA results across the three groups (practitioners, parents 
and administrators/higher education personnel) for each strand for the rating of 
recommended practice. A statistically significant difference was found only in the 
Child-Focused strand, (F = 5.065). A Tukey post hoc analysis revealed that practitioners 
(M-1.2196, SD— .2533) rated the items more favorably (p — .007) than parents 
(M— 1.3756, SD—. 3623). The effect size (Cohen’s d) for this difference was .4990, 
meaning that practitioners and parent ratings differed by almost one-half of a standard 
deviation unit. Administrators/higher education personnel (M= 1.2107; £D=2296) also 
rated child-focused items more favorably (p = .008) than parents (M- 1.3 756; ££>=.3623). 
The effect size (Cohen’s d) was .5437 meaning that on average, administrators/higher 
education personnel and parents differed in their ratings by more than half of a standard 
deviation unit. 
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Table 4 shows the results of analysis of variance across the three groups for each 
strand for the ratings of extent of use. As shown in Table 4, no main effects were found 
for any strands. 



Discussion 

As indicated above, the criterion for validation of each of the 250 practices was 
that more than 50% of the respondents to a particular practice would rate strongly agree 
or agree in response to whether it represents a recommended practice. All 250 of the 
practices met this criterion. It can be seen from Table 1 that there were similar patterns 
of responses across the strands. In every strand, strongly agree was the most frequent 
response followed by agree as the second most frequent response. The Family-Based 
strand had the largest strongly agree response at 74%, while the Technology strand had 
the smallest at 56%. Disagree and strongly disagree ratings were very infrequent in all 
strands. 

Examining Table 2, several things become apparent. Sometimes was the most 
frequent response to the question of extent Of current use for every strand. Also for each 
strand frequently and sometimes responses, taken together, accounted for more than 50% 
of all responses. Frequently was the second most frequent response for all strands with 
the exception of Technology and Policy and Systems Change where rarely was the 
second most frequent response. Undecided ratings and no rating (left blank) for four of 
the strands (Assessment, Child-Focused, Family-Based, and Interdisciplinary) was 5% to 
7% of the responses while for three of the strands (Technology, Policy and Systems 
Change, and Personnel) “undecided” and missing data was 14% to 16% of the ratings. 
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Unlike the study by Meyer et al (1987), there was a good deal of agreement 
across constituent groups in this field validation. A comparison of responses across 
groups of respondents resulted in a main effect for only the rating of recommended 
practice and for only one strand, Child-Focused. While all three groups had a majority of 
agree or strongly agree responses, the practitioners group and the administration/higher 
education group both rated the practices more favorably than the parents group. Effect 
sizes for both of these differences were close to or more than one-half of a standard 
deviation unit. It is not clear why the parent group did not rate these practices as strongly 
as the other two groups. It could be speculated that perhaps the terminology used in the 
strand was less familiar to the parents. 

In summary, all of the practices which resulted from the review of research and 
the focus groups were validated as recommended practices by the respondents surveyed. 
The data indicate a strong validation of the practices by all three of the respondent groups 
with a difference in the ratings found for only one of the seven strands. This is different 
from the validation of the original DEC Recommended Practices where differences were 
found in agreement for four of eleven strands (McLean & Odom, 1996). On average, the 
respondents reported seeing these practices in use sometimes and even frequently. There 
were no differences between groups for any of the seven strands relative to the rating of 
frequency of use. Again this is different from the validation of the original DEC 
Recommended Practices where differences were found in ten of the eleven strands 
(McLean & Odom, 1996). In summary, the results of this study support the social 
validity of the revised DEC Recommended Practices on the basis of stakeholder opinion. 
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Mean Rating 1 by Respondent Groups: 
To what extent do you see this practice? 
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(n = 199/388) 

Systems Change 2.1218 (SD= .4382) 2.1976 (SD= .6438) 2.1196 (SD= .4975) 

(n = 1 75/388) 
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Appendix D 

Products, Web Site, and Training 
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DEC Recommended Practices Series 




DEC Recommended Practices in Early Intervention/Early Childhood Special Education 



Birth to Age 5 

This resource bridges the gap between research and practice, offering much-needed 
guidance to parents and professionals who work with young children with disabilities. 
Recommended Practices outlines how families and educators can improve the develop- 
ment and learning outcomes for these children and the different meanings associated 
with those outcomes, including improved social competence, independence, problem 
solving, and enhanced family functioning. Recommended Practices covers various topic 
areas, including: Assessment; Child-Focused Practices; Family-Based Practices; , 

Interdisciplinary Models; Technology Applications; Policies, Procedures, and 
Systems Change; and Personnel Preparation. DCS143REC, $20.00 




DEC Recommended Practices Program Assessment: 



Improving Practices for Young Children With Special Needs and Their Families 




Birth to Age 5 

A companion to DEC Recommended Practices in Early Intervention/ Early Childhood 
Special Education, this resource will help you assess and improve the quality of serv- 
ices you provide to young children with disabilities and to their families. Learn to 
evaluate direct services and indirect supports, based on the recommended prac- 
tices; determine the strengths and needs of your program; and much more. 

Includes reproducible Program Assessment, Summary, and Action Planning forms. 
DCS143ASSESS, $20.00 



Co 













ijhji 



DEC Recommended Practices Video 

Selected Strategies for Teaching Young Children With Special Needs 
Ages 2-5 

This video demonstrates different learning environments and teaching proce 
dures from DEC Recommended Practices in Early Intervention/Early Childhood 

Special Education, and DEC Recommended Practices Program Assessment, including \ 4^ 

peer-mediated strategies, consequences, prompting strategies, naturalistic teach- \ 
ing procedures, and much more. These effective strategies are based on an exten- \ 
sive literature review and information gleaned from focus groups of parents, 
teachers, and administrators about what best promotes learning for young children 
with special needs. DCS143VID, $20.00 






Personnel Preparation in Early Childhood Special Education: Implementing the 
DEC Recommended Practices 

This publication provides guidance for faculty in higher education and for professional development specialists 
on the application of the DEC Recommended Practices in Personnel Preparation. Includes case studies from a 
variety of preservice and inservice programs that provide specific examples of implementation of the DEC 
Recommended Practices, (expected publication date: September 2002) 
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DEC Recommended Practices Series Order Form 




SOPRIS 

WEST 

Educational Services 



Fax Your Order Today to (888) 81 9-7767 Call for special large 

With faxed orders, please include VISA/MC information or signed Purchase Order. quantity discounts 



Product 

Code 


Product 

Title 


Qty. 


Unit Price 


Total 
^ Price 


DCS143REC 


DEC Recommended Practices (184 pages) 




$ 20.00 




DCS 143 ASSESS 


DEC Recommended Practices Program Assessment (136 pages) 




$ 20.00 




DCS143VID 


DEC Recommended Practices Video 




$ 20.00 




"Minimum order is S14.9S, plus S/H 
"♦Enter greater amount: 10% of subtotal or $4.00 
Tax Exempt # """Add applicable sales tax for your state 


Subtotal* 




Shipping/Handling** 




Sales Tax*** 




Total Amount Due 





PAID BY: 

Name 

job Title 

Grades You Work with 

School/Organization 

District and County 

Street Address 

City/State/Zip 

Country 

This address is my: □ Home O School/Organization 

Daytime Phone 

Fax 

E-mail 

May we e-mail you about these and related resources? □ Yes 



SHIP TO: 

Name 

)ob Title 

Grades You Work with 

School/Organization 

District and County 

Street Address 

City/State/Zip 

Country 

This address is my: □ Home □ School/Organization 

Daytime Phone 

Fax 



□ No (We do not sell, rent , or share your personal information with anyone.) 



Four Easy Ways to Order 



Fax Toll Free 
(888) 819-7767 



Phone Toll Free 
(800) 547-6747 



Mail to Sopris West 
4093 Specialty Place 
Longmont, CO 80504 



Internet 

www.sopriswest.com 



□ VISA □ MC 



□ Signed Purchase Order # _ 

(MUST be faxed or attached to order) 



□ Check/Money Order 

(Payable to Sopris West) 



tmber Expiration Date (Month/Year) 

n nn n-n nnn -nnnn- nnnn □□ / □□ 



Printed Name As It Appears on Card 



Cardholder Signature 



Sopris West Educational Services • Proven and Practical • (800)547-6747 



o 
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DEC Home Page 



Page 1 of 2 




The Division for Early Childhood 



The Division for Early Childhood (DEC) of the Council for Exceptional Children 
(CEC) is a nonprofit organization advocating for individuals who work with or on behalf 
of children with special needs, birth through age eight, and their families. Founded in 
1973, the Division is dedicated to promoting policies and practices that support families 
and enhance the optimal development of children. Children with special needs include 
those who have disabilities, developmental delays, are gifted/talented, and are at risk of 
future developmental problems. 



Effective July 1, 2002 

The DEC Executive Office's new contact information: 

Sarah A. Mulligan, Executive Director 
The Division for Early Childhood 
634 Eddy 

Missoula, Montana 59812-6696 

Phone 406.243.5898 
Fax 406.243.4730 
Email dec@selway.umt.edu 



• DEC News 



Nominate someone for a DEC Award! Deadline is September 9, 
2002. 

More information 



Nominate someone to serve on the DEC Executive Board. 

Deadline is September 9, 2002. 

More information 



DEC 2002 - The 18th Annual Conference 

on Young Children with Special Needs and Their Families 

December 5-8, 2002 San Diego, CA 
More information 



Conferences and Meetings 


/y 


DEC 

Recommended 






(fr 


Practices 





www.dec-sped.org/ 
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7/22/02 



DEC Home Page 



Page 2 of 2 



Young Exceptional 



Journal of Early 



m 















DEC 

Subdivisions 



DEC Executive 



Board, 
Committee 
Chairs & 
Editors 





DEC Policies, 

Position Statements & Concept 
Papers 




Join DEC/CEC 




Links to Other Sites 




IDEA 




U4w0fkTPfW«J 
FJodljb» Pt&grJu 



Publications 






tk 

fv : -r^V- ■ 



fir 

fSil 



Job Boards 




Awards 




Governmental 
Information 
& Children's 
Action 
Network 




Welcome to the Division for Early Childhood web site. As with most 
sites, we are under construction. We could really use your help and 
input with development. If you would like to help or get more 
information about DEC, please e-mail us by 'clicking' on Tell DEC 
below. 




The opinions and information contained in the articles , discussions f advertisements \ classifieds or linked 
linked web sites are those of the authors of that materia! and \ unless otherwise noted, not necessarily 
those of the Division for Early Childhood , nor does the Division for Early Childhood review , endorse or 
guarantee that their offerings are suitable or correct. Accordingly , the Division for Early Childhood 
assumes no liability , loss or risk which may be incurred from the material or as a consequence, directly 
or indirectly, of the use and application of any of the contents of those materials. 
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in Early Intervention/ 

Early Childhood Special Education 
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RECOMMENDED PRACTICES INITIATIVE 

The DEC Recommended Practices Project synthesized the 
knowledge found in the scientific/professional literature and the 
knowledge from experience of parents, practitioners and 
administrators about those practices that produce the best outcomes 
for children. This set of recommended early intervention/early 
childhood special education practices is now available in several formats including a book, a self- 
assessment guide and a video. Please see the Products section for more information. 
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Funded by the Office of Special Education Programs, U.S. Department of Education 



BRIDGING THE GAP INITIATIVE 







IThe Bridging the Gap Initiative was developed to provide a model 
to move the DEC Recommended Practices into actual practice in 
.;f order to implement and sustain high quality services to children and 
families. 



This model will facilitate the sustained use of the identified practices through training and systems 
change procedures. The model is being developed and implemented in two early childhood sites with 
the goal being to build the sites' capacity to implement the practices over time. The model, materials 
and outcomes at the sites are evaluated on an ongoing basis for improvement and revision. We will 
disseminate the model and findings through DEC and partner national organizations. 



PRODUCTS 



DEC Recommended Practices in Early Intervention/Early Childhood Special 
Education 

Sandall, McLean, Smith (Eds.) (2000) 
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This resource bridges the gap between research and practice, offering much- 
needed guidance to parents and professionals who work with young children with 
disabilities. Recommended Practices outlines how families and educators can 
improve the development and learning outcomes for these children and the 
different meanings associated with those outcomes, including improved social 
competence, independence, problem solving, and enhanced family functioning. 
Recommended Practices covers the following seven topic areas: 




• Assessment - Neisworth & Bagnato 

• Child-Focused Interventions - Wolery 

• Family-Based Practices - Dunst & Trivette 

• Interdisciplinary Models - McWilliam 

• Personnel Preparation - Stayton & Miller 

• Policies, Procedures, and Systems Change - Harbin & Salisbury 

• Technology Applications - Stremel 

To order, please see Sopris West's website at wwv.sonriswest.com or call 800-547-6747. 
Cost: $20.00 



NEW! 

DEC Recommended Practices Program Assessment: Improving Practices for Young 
Children With Special Needs and Their Families 

Hem meter, Joseph, Smith, Sandall (Eds.) (2001) rl. 



Ages Birth through Age 5 



rtfcfoT 

lakes 






A companion to DEC Recommended Practices in Early Intervention/Early 
Childhood Special Education, this resource will help you assess and S " 5 : " 

improve the quality of services you provide to young children with HP %-C 

disabilities and to their families. The assessment is appropriate for Head 
Start, child-care centers, public schools, or other early childhood programs that implement 
developmental ly appropriate classrooms and curricula. 




Use this assessment to: 



• Evaluate direct services and indirect supports, based on the recommended practices. 

• Determine the strengths and needs of your program. 

• Evaluate the impact of training, technical assistance, and other interventions. 

• Identify professional development needs of staff. 

Includes reproducible Program Assessment, Summary, and Action Planning forms. 

To order, please see Sopris West's website at www.sopriswest.com or call 800-547-6747. 
Cost: $20.00 



NEW! 

DEC Recommended Practices Video: Selected Strategies for Teaching 
Young Children With Special Needs 

BEST COPY AVAILABLE 



RJC ttp://www.dec-sped.org/rppage.html 




7/22/02 




Recommended Practices Page 



Page 3 of 5 



Ages 2 through 5 

This video demonstrates different learning environments and several 
teaching procedures from DEC Recommended Practices in Early 
Intervention/Early Childhood Special Education, and DEC Recommended 
Practices Program Assessment, including the use of: 

• peer-mediated strategies 

• consequences 

• prompting strategies 

• environments that promote learning 

These effective strategies are based on an extensive literature review and information gleaned from 
focus groups of parents, teachers, and administrators about what best promotes learning for young 
children with special needs. 

To order, please see Sopris West's website at www.sopriswest.com or call 800-547-6747. 

Cost: $20.00 




Personnel Preparation in Early Childhood Special Education: Implementing the DEC 
Recommended Practices 

This publication provides guidance for faculty in high education and for professional development 
specialists on the application of the DEC Recommended Practices in Personnel Preparation. Includes 
case studies from a variety of preservice and inservice programs that provide specific examples of 
implementation of the DEC Recommended Practices. 

(Expected publication date: September 2002) 



REPRODUCIBLE CHECKLISTS 

The following checklists are offered as examples of ways that individuals or teams might use the 
Recommended Practices to guide program development or evaluation. Readers are encouraged to 
design their own checklists or discussion guides that focus on their particular program development 
needs and concerns. The checklists come from DEC Recommended Practices in Early 
Intervention/Early Childhood Special Education (Sandall, McLean and Smith, 2000). 

These checklists are in Adobe PDF format and can be downloaded and copied without permission - 
distribution is encouraged! 




Parent Checklist 




Self-Assessment: Child-Focused Interventions 




Administrator's Essentials 



'www.dec-sped.org/rppage.html 



BEST COPY AVAILABLE 
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DEC RECOMMENDED PRACTICES 
LITERATURE SOURCES 

Research published in peer-reviewed professional journals from 1990 through 1998 was included in 
the literature review. Forty-eight journals relevant to the field of EI/ECSE were selected and 
reviewed (list of journals) . The criteria used to determine if an article would be included in the study 
were that the article (a) must be an original research report and (b) involved children with disabilities, 
birth through 5, families of children with disabilities birth through 5, personnel who serve them, or 
policies and systems change procedures that support effective practice with this population. Each 
article that met these criteria was analyzed for major aspects of research design and results and 
assigned to a category or "strand" of practice. Some articles were determined by reviewers to support 
more than one strand. 

The literature review along with the results of focus groups of parents, practitioners and 
administrators produced a thoroughly supported synthesis of research and experienced-based 
knowledge about EI/ECSE practices for young children with disabilities birth through 5 and their 
families. This synthesis of evidence-based practices resulted in the 240 DEC Recommended 
Practices. The references for the articles included in the synthesis are offered here and are categorized 
by the "strand" of practice they were determined by reviewers to relate to. 

• List of all Journals 



Literature Lists: 

• Assessment Strand 

• Child-Focused Practices Strand 

• Famil y-Based Practices Strand 

• Interdisciplinary Models Strand 

• Technology Applications Strand 

• Personnel Preparation Strand 

• Policies, Procedures and Systems Change Strand 



DEC RECOMMENDED PRACTICES CONSULTANT BANK 



As a service to the field, DEC has created a Recommended Practices Consultant Bank. The members 
of this group have been nominated by the Recommended Practices Strand Chairs as experts in a 
specific Recommended Practices strand. These consultants may be able to provide training and 
technical assistance on the DEC Recommended Practices on a particular topic. 



DEC's only involvment is providing these names. Fees and specific arrangements must be made 
directly between the requester and the consultant as a private consultation request. Please choose one 
of the following strands to view the list of consultants for that topic: 



• Assessment 

• Child-Focused Practices 

• Family-Based Practices 

• Interdisciplinary Models 

• Technology Applications 

• Personnel Preparation 

• Policies. Procedures and Systems Change 
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DEC encourages the exchange of diverse opinions. The consultants are expressing their own opinions and expressions of 
fact which are not necessarily those of DEC, nor have they been endorsed or reviewed for accuracy by DEC. 

Accordingly, DEC assumes no liability, loss or risk which may be incurred as a consequence, directly or indirectly, to the 
use and application of any such facts and opinions. 
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Recommended Practices/Bridging the Gap National and Regional* Workshops 



Date 


Location 


# of participants 


April 1999 


Charlotte (CEC Conference) 


approx. 50 


December 1999 


Washington (DEC Conference) 


approx. 150 


April 2000 


Vancouver (CEC Conference) 


100 


November 2000 


Atlanta (NAEYC Conference) 


100 


December 2000 


Albuquerque (DEC Conference) 


100 


May 2001 


Orlando (National Head Start Association) 


75 


November 2001 


Anaheim (NAEYC Conference) 


75 


November 2001 


Tampa (CEC/TED) 


75 


December 2001 


Boston (DEC Conference) 


50 


April 2002 


New York (CEC Conference) 


45 


April 2002 


Seattle (regional) 


20 


June 2002 


Worcester (regional 


90 


June 2002 


Ogden (regional) 


200 



* In addition to national arid regional workshops, project staff presented workshops in 
several states. 
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Focus Group Recommendations 
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ADMINISTRATOR recommended prac tices focus group 

Services to Families 

• Administrators support FLEX scheduling to enable EL/ECSE providers to 
accommodate family schedules 

• Infrastructure has mission statement which reflects family-centered practices and 
values 

• Administrators engage in shared decision making with families 

• Administrators support service delivery models that are different from school-age 
programs i.e. teachers working in the home 

• System has funding to support home visits 

• Alternative dispute resolution processes are implemented 

• Administrators support practitioners in using adult learning principles 

• Administrators facilitate a balance of respecting family-identified outcomes and 
professional opinion 

• Administrative structure supports mentoring/ supervision of staff 

Services to the child 

• Programs have appropriately trained and certified staff 

• Administrative structures support hiring and retention of appropriately trained and 
certified staff 
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• Administrators are knowledgeable about and can assist access to community 
resources for children 

• Administrators ensure that appropriate 1EP/IFSP procedures are followed 

• Development of EFSPs/IEP's are outcome-based rather than system based 

Cultural/Linguistic 

• Administrators ensure availability of translators/interpreters and bilingual programs 

• Administrators ensure cultural competence in service delivery through training, staff 
recruitment, and collaborative efforts with the community 

Technology 

• System ensures that staff have appropriate technology and training related to the use 
of the technology 

• System ensures that children have appropriate assistive technology 

Learning Environments 

• Administrators ensure that the physical environment supports development and 
learning 

• Administrative structures support developmentally appropriate environments 

• Administrators ensure that services are available in a variety of settings in the 
community 

• Administrators work to ensure the safety of staff who do home visits 

Interdisciplinary Models 

• System provides adequate training, time and resources for teaming. 

• Interdis/transdis models aren't a substitute for adequate numbers of personnel 

Policy and Procedures 
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• System provides adequate fiscal resources to support "appropriate" numbers of 
supervisors/staff (i.e. ratios) and other administrative duties 

• Programs have mission statements, periodically set goals and outcomes and perform 
on-going evaluation of the outcomes 

• Resources and procedures are available for evaluation 

• Administrators facilitate shared decision-making with families, staff, etc. 

• Administrative structure has funding to support services i.e. team meetings, travel, in- 
service consultation, in-service, paperwork 

• Administrative structure provides support for alternative funding structures: 

Medicaid waiver, child-care subsidies, blending funds across programs 

• The goals of the IFSP/IEP determine the types/amount of services needed i.e. 
outcomes drive personnel needed 

Personnel Development 

• Program coordinators/supervisors have training in Early Childhood Education, Early 
Intervention, Early Childhood Special Education and supervision 

• Administration supports "learning" (e g., inservice, coursework, time etc.) 

• Administrators have knowledge of how to access alternative funding structures 

• Administrators support on-going staff development plans 

• Administrators are affiliated with professional early childhood organizations, 
encourage staff to maintain their affiliations, and support staff attendance at meetings 
and conferences to enhance professional growth. 

Systems Change 

• Administrative structure supports adequate child-staff ratio, group size and case loads 
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• Interagency agreements are developed, implemented and evaluated on a continual 
basis 

• Structures are in place that support local interagency councils/agreements in 
communities 

• A quality assurance system is in place. Administrators have goals and objectives for 
program improvement 

• Parents and other stakeholders have input into the development and implementation 
of evaluation systems 
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Indicators of Recommended Practices: 
Family Perspectives 



Professionals’ communication, both verbal and written, conveys that professionals 
consider family members as equal partners in all phases of the early intervention process. 

Program personnel routinely provide opportunities for adults who have experienced early 
intervention to exchange information with families. 

Personnel preparation professionals routinely provide opportunities for adults who have 
experienced early intervention to be involved in planning and delivering preservice and 
inservice education. 

Program administrators ensure that staff development opportunities address practitioners 
understanding and attitudes regarding the supports and needs of young children with 
disabilities and their families. 

Program personnel provide consistent opportunities for families to share information and 
support. 

Program personnel provide each child and family with the necessary services to support 
full participation in inclusive and natural environments. 

Professionals work collaboratively with families to identify how each environment, (e.g., 
home environment, day care environment, early intervention environment, and community 
environments) promotes or hinders the development and learning of each child. 

Professionals work collaboratively with families to develop and use a shared terminology 
during all phases of the intervention process. 

Professionals work collaboratively with families to plan early for transitions into and out 
of different family-valued service systems and environments. 

Professionals work collaboratively with families to identify and understand the policies and 
procedures that govern reimbursement for service delivery. 



1 1 . Professionals work collaboratively with families to identify and remove reimbursement and 
related barriers to desired service delivery, for example Medicaid reimbursement. 

12. Professionals share and explore with families a full range of placement options with family 
preferences guiding where services are delivered. 

13. Professionals work jointly with to routinely monitor child placements in response to 
changes in child and family needs and circumstances. 

14. Professionals work collaboratively with families to identify and access opportunities that 
promote full participation and membership across multiple community and leisure settings. 

15. Families and professionals work together to identify and provide the types and levels of 
support required for child success across environments. 

16. Professionals provide families with the information necessary for families to be informed 
decision makers. 

17. Professional decision-makers and policy developers set an administrative climate that 
encourages practitioner and family collaboration. 

18. Professionals and families work collaboratively to explore all service options and clarify 
family preferences in the development of the EFSP and IEP. 

19. Professionals, families and administrators work together to create a climate where 
practitioners feel comfortable in sharing their perspectives, even if professional 
perspectives differ from family perspectives without usurping family decision making. 

20. Program personnel provide timely information to families that prepares them for eligibility 
and service system differences between Part C and Part B. 

21 . Program administrators ensure that individuals with disabilities participate in developing, 
delivering and evaluating services to young children with disabilities and their families, for 
example, by serving on advisory councils, as paid employees, and as mentors to young 
children with disabilities and their families. 

22. Professionals and families work together to identify multiple outcomes of program 
effectiveness including objective indicators of child change, subjective views of increased 
membership and community participation , and overall family satisfaction. 

23. Families and professionals work together to ensure that family-centered principles and 
practices are consistent across Part C and Part B. 
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Program personnel engage in multidisciplinary and interagency planning and decision 
making collaboratively with families. 

Personnel preparation professionals work collaboratively with families and practitioners in 
the education of current and prospective personnel. 

Personnel preparation professionals utilize active strategies consistent with the principles 
of adult learning in the education of current and prospective personnel. 

Personnel preparation professionals employ strategies designed to identify personal values 
and biases as part of the education of current and prospective personnel. 

Personnel preparation professionals plan and provide follow-up support within service 
delivery contexts as a key component of the education of current and prospective 
personnel. 

Families and professionals work together to access, contribute to, and benefit from current 
and emerging information and communication technology. 

Professionals and families collaboratively identify the individual strengths of children and 
families as the basis for planning and implementing services and supports. 

Professionals and families collaborate to identify effective strategies for educating 
administrators and policy makers on their roles in creating climates that support family- 
centered practices. 



Recommended Practices: Practitioners Focus Group 

What practices are related to better results for young children with 
disabilities, their families, and the personnel who serve them? 

General Comments 

1. Program directors should ensure that service coordination is provided for 
children and families at the birth to three and the preschool level. 

2. Practitioners provide services for children in their natural environments. 

3. A single professional/practitioner coordinates services for children and 
families through the preschool years. 

4. Professionals define natural environments in ways that are responsive to 
family beliefs, needs, and concerns and are consistent with families’ lives. 

5. Practitioners offer families program options and descriptions of those 
options from which families can define and determine their natural 
environments. 

6. Professionals who provide preservice and inservice preparation should 
encourage awareness and knowledge of a wide range of options for service 
delivery, settings, and arrangements. 

7. Practitioners should plan for and provide IEP/IFSP processes that allow 
for open discussion of the family’s usual routines and the possibilities for 
service delivery options and settings that are consistent with or complement 
family routines. 

8. Practitioners should collaborate across all agencies who can and do 
provide services to children and families. 

9. Program personnel provide advocacy and support services (i.e., a helpful 
and knowledgeable person) that are available to families before, during, and 
after transition points (before age 3, into kindergarten, into first grade/primary 
class). 
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10. Program personnel have ongoing systems change mechanisms in place in 
order to be responsive to families and communities. 

1 1 . Professionals in preservice and inservice preparation should include 
information on family centered care. 

12. Professionals in preservice and inservice preparation should include 
families in a variety of ways in their preparation programs. 

13. Practitioners maintain and expand their skills and knowledge by taking 
continuing education courses and other learning experiences. 

14. Practitioners provide families with complete and unbiased information so 
that they can make informed choices. Choices are genuine and choices are 
respected. 

15. Program personnel provide staff development activities that include self 
evaluation (of individual needs and program needs), incorporate principles of 
adult learning, and encourage reflective practice. 

16. Program personnel have systems of accountability in place for individual 
staff members and the program as a whole. 

17. Program personnel use program standards. 

18. Program personnel monitor use of program standards. 

19. Program personnel strive for accreditation of their program through DEC 
(or other accrediting agency/organization). 

20. State personnel (or other appropriate individuals/teams) evaluate 
programs against program standards on a regular basis. 

21 . Practitioners demonstrate skills and knowledge that are appropriate to the 
age group that they serve. 

22. State personnel ensure that a coordinated and collaborative system of 
services is in place for children ages birth -five and their families. 



Transition Recommendations 



1 Practitioners provide families with current and accurate information about 
the next environment and services. 

2. State or local program personnel develop policy that allows transitions in 
services and settings be made in response to child and family needs rather 
than solely in response to the child’s 3rd birthday. 

3. Program personnel develop policy that ensures that child assessments for 
the purposes of continuing eligibility or transition are done by individuals and 
teams who know the child and family . 

4. Receiving teachers and other team members should observe the child in 
familiar settings prior to transition. 

5. Program personnel schedule placement meetings or activities so that 
sending and receiving teams and families have the opportunity to observe and 
to learn about each other’s settings and expectations before the child changes 
programs. 

6. Program personnel demonstrate family centered care and respect 
throughout the birth - five system(s) of services. 

7. State and local program personnel develop policy that allow transitions to 
occur at times that are consistent with child, family, and community schedules 
and rhythms. 

8. State and local program personnel develop funding policies that are 
flexible and allow children and families to complete school years or other 
naturally occurring schedules. 

9. State and local program personnel offer a range of service delivery options 
including itinerant services, inclusive programs, and so forth. 

Service Delivery Options for 3-5 

1 . Program personnel provide service delivery options for preschoolers that 
are flexible and responsive to child and family needs. 
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2. Program personnel provide classroom options for preschoolers that 
include a mix of ages and abilities. 

3. Program personnel provide classrooms that are of a class size and teacher- 
child ratio that are based on sound research. 

4. Practitioners plan for and provide opportunities for children to be 
meaningful members of their classrooms and the larger community. 

5. Practitioners use strategies that support membership and inclusion (i.e., 
minimal use of ‘pull out” services, paraeducators support the whole 
classroom, itinerants follow the community preschool’s schedule and 
calendar, etc.) 

6. All early childhood practitioners (general early childhood teachers, early 
childhood special educators, paraeducators) participate in joint training and 
other staff development activities. 

7. Program personnel ensure that all team members have adequate time for 
joint planning and problem-solving. 

8. State and local program personnel ensure that a range of service delivery 
options and settings is available and that these are consistent with principles 
of natural environments/least restrictive environment. 

Paraeducator Training 

1. Paraeducators participate in training opportunities that allow for 
continuing development of skills and knowledge (i.e., tiered or leveled 
training). 

2. Paraeducators have opportunities for increased job independence based on 
increased training. 

3. Program personnel provide clear job descriptions (roles, responsibilities, 
and working relationships). 

4. Paraeducators have opportunities for advancement. 
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5. Teachers and other service providers use effective and appropriate 
strategies for supervising paraeducators. 

6. Professionals in preservice and inservice preparation deliver coursework 
or other trainings to help teachers supervise paraeducators. 

7. Paraeducators follows standards or competencies. 

8. State and local program personnel develop and use standards or 
competencies for paraeducators as a system of accountability. 

IEP/IFSP 

1 . Practitioners use the IEP/IFSP on a regular and frequent basis. 

2. Team members write goals and objectives that are meaningful (represent 
worthwhile child and family outcomes). 

3. Team members use measurement systems that allow for measurement of 
meaningful growth and change. 

4. Practitioners recognize the family’s knowledge and experience in 
developing the IEP/IFSP. 

5. Practitioners monitor children’s progress on goals and objectives on a 
regular and frequent basis. 

6. Team members ensure that program and placement options are discussed 
and considered as part of the IEP/IFSP process. 

7. All team members participate in the IEP/IFSP process. 

8. Team members write goals and objectives that are rOutines-based, 
functional, and child oriented. 



Supports to Families 



1 . Practitioners provide families with information and access to family 
support activities through the program and/or other community resources. 

2. Service coordinators meet and fulfill the roles and responsibilities of the 
service coordinator (i.e., not a specific person or discipline). 

3. Program personnel encourage meaningful participation of family members 
through a variety of opportunities and experiences including employment 
possibilities. 

Assessment 

1 . Practitioners assess children in contexts that are familiar to the child. 

2. Practitioners assess children after they have become familiar to the child. 

3. Practitioners assess children using tools that are matched to the purpose of 
the assessment. 

4. Families participate in the assessment process. (There are a range of 
options for participation.) 

5. Practitioners may use clinical judgment as part of the assessment. 

6. Practitioners use a variety of assessment tools and approaches including 
direct testing, family report, observation, and so forth. 

7. Practitioners select assessment tools and approaches that are appropriate 
to the child’s abilities and the family’s priorities. 

Other 

1 . Federal, state, and local program personnel ensure that EI/ECSE is 
adequately funded. 

2. Federal, state, and local program personnel develop funding policies that 
support services and service arrangements that enhance child and family 
functioning. 



3. Federal, state, and local program personnel ensure that practitioner’s 
caseloads are manageable. 

4. Federal, state, and local program personnel provide funding for adaptive 
equipment, computers, and other technology. 

5. Practitioners use instructional and therapeutic strategies that are 
appropriate, effective (data based), and matched to child and family needs. 

6. Practitioners work as team members in delivering EI/ECSE services. 

7. Professionals in preservice and inservice preparation provide training in 
team building and teamwork. 

8. Federal, state, and local program personnel provide or obtain funding for 
transportation for families as needed. 
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NOTICE 

Reproduction Basis 



This document is covered by a signed "Reproduction Release (Blanket)" 
form (on file within the ERIC system), encompassing all or classes of 
documents from its source organization and, therefore, does not require a 
"Specific Document" Release form. 



This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may be 
reproduced by ERIC without a signed Reproduction Release form (either 
"Specific Document" or "Blanket"). 



i rjk } ff -° 89 ( i/2003 ) 




